{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature Selection\n",
    "\n",
    "## Subset Selection\n",
    "\n",
    "<p>Probably the most common approach to explicit feature selection is to choose a subset of $k$ features from the total set of $p$ features. The idea scenario would be to consider every possible subset of size $k$ and choose the one that has the best out-of-sample error. Although ideal in theory, in practice, there are $\\binom{p}{k} = \\frac{p!}{k!(p-k)!}$ possible subsets of size $k$. Testing each one is often cost prohibitive when $p$ gets large.<br><br>\n",
    "\n",
    "<b><u>Forward Stepwise Selection</u></b><br>\n",
    "Forward stepwise selection uses a greedy procedure to learn a good subset of $k$ features (though not guaranteed to be the \"best\" subset). The procedure goes as follows:<br>\n",
    "<ul>\n",
    "    <li>1. Initialize: Curr_Best_Subset = { } (i.e., just the intercept)</li>\n",
    "    <li>2. Loop through each feature $j$:</li>\n",
    "        <ul>\n",
    "            <li> a. Add the feature to the current best subset, train a model</li>\n",
    "            <li> b. Get out-of-sample error on the model trained with Curr_Best_Subset+feature $j$</li>\n",
    "        </ul>\n",
    "    <li>3. Choose the feature with the best out-of-sample improvement in error </li>\n",
    "    <li>4. Add this best feature to Curr_Best_Subset, and log the out-of-sample error</li>\n",
    "    <li>5. Repeat steps 2 - 4 until stopping criterion is met.</li>\n",
    "</ul><br>\n",
    "Some possible stopping criteria are:<br>\n",
    "<ul>\n",
    "    <li>Stop at a pre-defined $k$</li>\n",
    "    <li>Stop when (Performance: Best $k$ - Performance: Best $k-1$)$<\\delta$ </li>\n",
    "    <li>Same as above, but using a 1-std error rule or a t-test to compare best $k$ vs. best $k-1$</li>\n",
    "</ul><br>\n",
    "It might not always make sense or be optimal to choose $k$ in advance. The other two stopping criteria amount to continually adding features until the feature does not improve the model, either in an absolute sense or using some statistical test to compare the difference between cross-validated means.<br>\n",
    "\n",
    "\n",
    "\n",
    "The following code shows how this can be done. We'll use the churn prediction data. For a metric, let's assume we care the most about having a good estimate of $P(Y|X)$, so we'll use logistic regression with log-loss as our feature selection criterion. Again, the log-loss is defined as:<br><br>\n",
    "<center>$logloss = -\\frac{1}{n} \\sum\\limits_{i=1}^n \\: y_i*log(p_i)+(1-y_i)*log(1-p_i)$</center><br>\n",
    "We'll also use the last stopping criteria shown above.\n",
    "\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Load the data. Not much prep is need since its a clean dataset\n",
    "import numpy as np\n",
    "import math\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import sys\n",
    "import course_utils as bd\n",
    "from sklearn import linear_model\n",
    "\n",
    "\n",
    "f = '../data/Cell2Cell_data.csv'\n",
    "dat=pd.read_csv(f, header=0, sep=',')\n",
    "dat['r'] = np.random.random(dat.shape[0])\n",
    "dat = dat.sort_values(by='r').reset_index(drop=True)\n",
    "dat = dat.drop('r', axis=1)\n",
    "\n",
    "train, test = bd.trainTest(dat, 0.5)\n",
    "lab = 'churndep'\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import KFold\n",
    "\n",
    "\n",
    "def runXVal_LogLoss(cv, X_sub, Y_tr):\n",
    "    '''\n",
    "    Runs LR cross validation with no regularization, returns mean and standard error of mean\n",
    "    '''\n",
    "    ll = []; \n",
    "    for train_index, test_index in cv.split(X_sub):\n",
    "        X_tr_f = X_sub.iloc[train_index]\n",
    "        X_va_f = X_sub.iloc[test_index]\n",
    "        Y_tr_f = Y_tr.iloc[train_index]\n",
    "        Y_va_f = Y_tr.iloc[test_index]\n",
    "\n",
    "        lr = linear_model.LogisticRegression()\n",
    "        lr.fit(X_tr_f, Y_tr_f)\n",
    "        P = lr.predict_proba(X_va_f)[:,1]\n",
    "        \n",
    "        ll.append(-1*(((Y_va_f==1)*np.log(P)+(Y_va_f==0)*np.log(1-P)).mean()))\n",
    "\n",
    "    return [np.array(ll).mean(), np.array(ll).std()/np.sqrt(len(ll))]\n",
    "        \n",
    "def LrForward_LogLoss(X_tr, Y_tr, cv):\n",
    "    '''\n",
    "    Runs cross-validated stepwise selection\n",
    "    Does not pick the best features, but returns data\n",
    "    Returns a dictionary that shows at each k: [feature set], x-validated mean, x-validated var \n",
    "    For each loop, chooses the feature with best mean+1stderr\n",
    "    '''\n",
    "    results = {}\n",
    "    curr_best = set([])\n",
    "    cand_list = set(X_tr.columns.values)\n",
    "    k = 1\n",
    "    \n",
    "    while (len(cand_list)>0):\n",
    "        best_mu = 10**10; best_serr = 10**10; \n",
    "        for f in cand_list:\n",
    "            use_x = list(curr_best)+[f]\n",
    "            mu, serr = runXVal_LogLoss(cv, X_tr[use_x], Y_tr)\n",
    "            if ((mu + serr) < (best_mu + best_serr)):\n",
    "                best_mu = mu\n",
    "                best_serr = serr\n",
    "                best_f = f\n",
    "        curr_best.add(best_f) #Add the best feature to the curr_best_set\n",
    "        cand_list = cand_list.difference(curr_best) #Remove the best feature from the candidate set\n",
    "        results[k] = [list(curr_best), best_mu, best_serr]\n",
    "        k+=1\n",
    "        \n",
    "    return results\n",
    "            "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Run the forward selection\n",
    "cv = KFold(n_splits=10)\n",
    "r = LrForward_LogLoss(train.drop(lab,1), train[lab], cv)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>\n",
    "\n",
    "\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEWCAYAAABMoxE0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3Xd4VFX6wPHvm0JCDy0iRAgWkN6V\nEGlSBFYFXETAXlCXRYqKuD9XYdFdWRdZUHYXRVdcRZQiiHQh1FCko4AiHQSEIEgNgeT9/XFuwhCS\nmSGZySSZ83me+2Tmzrn3vhPCvHPKPUdUFcuyLMvKqZBAB2BZlmUVbDaRWJZlWbliE4llWZaVKzaR\nWJZlWbliE4llWZaVKzaRWJZlWbliE4mVJ0Rkrog8Gug4Ak1EYkVERSQs0LHkVyKyRESe8sN57d+g\nn9hEUgiJyB0islJEfhORX0UkUUSaOq89JiIr8jomVe2kqh/78pzOB06yiJxx2eJ8eY28JiJ7ReR8\npvdUKZfnbC0iB30Vo5fXjBGRaSKS5Pwdfi8ij+Xh9YeJyKeu+/zxN2gZ9ltRISMipYBZwB+AyUAR\noAVwIZBx+VE/Vf0gpweLiACiqmk+jCn93GGqeikHh96jqgt9HU9O5fB9fAJsBqpi/vbqAhV9HZuV\nP9gaSeFTHUBVJ6lqqqqeV9UFqrpFRGoC44A455vuSQARiRCRkSKyX0R+EZFxIlLUea21iBwUkf9z\nvl3uFZEHndeqichJEQlxno8XkaPpgYjIJyIy0Hmc0VwhIjeLyFLnm2qSiHzhcsytIvKNU5P6UUR6\n5OSXICLNRWStc421ItLc5bUlIvJXEUkEzgFPish3Lq9/IyJrXZ4vF5GuzuOXRWSXiJwWkW0i0s2l\n3GNO7e+fInIcGCYioc7vNklEdgO/y8n7cc7fzKlpnhSRzSLS2uW1x0VkuxPXbhF5xtlfHJgLVHKt\n4YjIBBF5w+X4K2otzr/zEBHZApwVkTDnuGkickxE9ohIfzfhNgUmqOpZVb2kqhtVda437yWL9/2E\n895OiMh8Eanq8lptl7+XX5y/047A/wEPOO93s1PW9W8wRET+LCL7ROSoiPxPREo7r6U3Pz7q/J9I\nEpFXvPpHClaqardCtAGlgOPAx0AnoEym1x8DVmTa909gJlAWKAl8DbzpvNYauASMAiKAVsBZoIbz\n+n6gsfP4R2A3UNPltYbO4yXAU87jScArmC8ykcAdzv7iwAHgcUxtuSGQBNTK5r1mnDPT/rLACeBh\n5zy9nOflXI7bD9R2Xi8JJAPlgXDgF+BnZ39R4LzLsfcDlZzYH3B+F9e7/G4vAc855y0KPAv8ANzg\nxLUYUCAsm/e0F2iXxf7Kzr9rZ+fa7Z3nFZzXfwfcBIjzb3QOaOTyb3gw0/kmAG+4PL+ijBPHJifu\nos411wOvYWq5Nzr/1ndl8z4WAolAT6DKNb4X17+VLsBOoKbzO/0zsNJ5rSRwGHgB83dUErjdeW0Y\n8Gl2fy/AE855bwRKAF8CnzivxTr/RuOd914fU6uqGej/3/l1szWSQkZVTwF3cPk/wjERmSki12VV\nXkQEeBoYpKq/qupp4G+YDwBXr6rqBVVdCswG0msKS4FWIpLebDHVeV4Nk9Q2Z3HZi5gmj0qqmqyq\n6X02dwN7VfUjdb7FAtMwH97Zecf5VntSRDY4+34H/KSqnzjnmYT5ML/H5bgJqrrVef00sBZoCTR2\nYk4E4oFmzrmOA6jqFFU9pKppqvoF8BNwm8t5D6nqu855zzu/p9GqekBVfwXedPNe0s1weU8znH0P\nAXNUdY5z7W+AdZgPY1R1tqruUmMpsADTpJkb7zhxn8fUMCqo6nBVTVHV3Zi/r8x/J+nuB5YDrwJ7\nRGSTOP10nt5LJs9ivtRsV9O89jeggVMruRs4oqpvO39Hp1V1jZfv7UFglKruVtUzwJ+AnnLlIIi/\nqKnRb8b8TdT38txBxyaSQsj5T/eYqsYAdTDfoEdnU7wCUAxYn/7hBcxz9qc7oapnXZ7vc84JJpG0\nxnwIL8N862vlbMs1676HlzDfnL8Vka0i8oSzvypwu8uH6EnMf3h3bev9VTXK2Ro5+yo5Mbrah/km\nnO5Aptdd38fSTO9jaXohEXnE+VBMj68OpiaT3XkrZdqXOa6sdHV5T12dfVWB+zP9bu4Arnfi6iQi\nq50mnpOYD+XyWZ/ea65xV8U0j7le//+ALL+gqOoJVX1ZVWs7ZTZhEqR4ei+ZVAXGuJT7FfO3UxlT\nW9qVw/eW+W9kH6bG4/p+jrg8PoepuVhZsJ3thZyq/iAiE4Bn0ndlKpKEabqprao/Z3OaMiJS3CWZ\nVAG+dx4vBf4BHHQer8D0wyTj8gGcKaYjQB8wI8yAhSKyDPPBtVRV21/Tm7zaIcwHkKsqmASZEUam\n15cCb2OavEZgmsLGY5o0/uXEWtXZ1xZYpaqpIrIJ88GW3XkPYz7wXOPIiQOYppc+mV8QkQhMze0R\n4CtVvejUZNLjymqK77OYLxDpskrWrscdAPao6i3XGriqJonISOBRTPNetu8lCweAv6rqxMwvOP8e\n2dWIPE1rnvlvpAqmWfIXIMaLuCwXtkZSyIjprH5BRGKc5zdg+ghWO0V+AWJEpAiAU2MYD/xTRKKd\nYyqLyF2ZTv0XESkiIi0wTQpTnON/wiSihzBJ4JRzjd+TTSIRkfvT48N8YCuQhhltVl1EHhaRcGdr\nKmaQwLWY45ynt9NJ/ABQyzl/dlYCNTDNVN+q6lacGhKmpgWmD0eBY877eBxTI3FnMtBfzHDYMsDL\n1/he0n0K3CMid4npwI90OshjMH0WEU5cl0SkE9DB5dhfgHLpncmOTUBnESnrNEsO9HD9b4HTTgd8\nUSeGOi7NVVcQkb87r4eJSEnMKMKdThOhu/eS2TjgTyJS2zlvaRFJb+qcBVwvIgPFDBgpKSK3u7zn\nWHEGgmRhEjBIzICREpgmsy80Z6Psgp5NJIXPacyH3xoROYtJIN9jOiQBEoCtwBERSXL2DcF0PK4W\nkVOYjtIaLuc8gvnAPwRMBJ5V1R9cXl8KHFfVAy7PBdhA1po68Z3BdPIPcNqqT2M+AHs61zoC/B3z\nIek158Pqbuc9H8c0pd2tqklujjnrxLtVVVOc3auAfap61CmzDVNrWYX5oKqL6UtxZzwwH9PGvgHT\nqXvNnN9tF0xz0jHMN/XBQIjze+uPSVongN6Y32v6sT9gPjh3O01Elbg8PHcvpj8lY+RcNtdPxfxO\nGwB7MDXZD4DS2RxSDJgOnMR0ylcF7vX0XrK47nTM38Dnzt/m95hBJDjvuz2m7+sIpr+qjXPoFOfn\ncZe+M1f/dX4Hy5z3k4wZJGHlgKjaha2s7IkZlvmp099iWZZ1FVsjsSzLsnLFJhLLsiwrV2zTlmVZ\nlpUrtkZiWZZl5UpQ3EdSvnx5jY2NDXQYlmVZBcr69euTVLWCp3JBkUhiY2NZt25doMOwLMsqUETE\nm5kYbNOWZVmWlTs2kViWZVm5YhOJZVmWlStB0UdiBbeLFy9y8OBBkpOTAx2KZeVLkZGRxMTEEB4e\nnqPjbSKxCr2DBw9SsmRJYmNjMbOYW5aVTlU5fvw4Bw8epFq1ajk6R1A0bR06dCjQIVgBlJycTLly\n5WwSsawsiAjlypXLVY09KBLJ4cOHAx2CFWA2iVhW9nL7/yMoEollWZblP0GTSEQEEWHYsGGBDsUK\nQiVKXL1K67Bhwxg5cmSOzjdlyhRq1qxJmzZtrtiflpZG//79qVOnDnXr1qVp06bs2bPH7blGjx7N\nuXPnsnwtNjaWpKRsl3EpsJ544gmio6OpU8fTumRX2rRpE3PmzMn29fzy+8rrOIImkWzZsgVVtYnE\nKhQ+/PBDxo8fz+LFi6/Y/8UXX3Do0CG2bNnCd999x/Tp04mKinJ7LneJpLB67LHHmDdvnueCmXhK\nJLmRmpp6xfNLl7JfrNHda4EQNIlk+PDhgQ7BKkBWrYI33zQ/A2nSpEnUrVuXOnXqMGTIEMD8La9Y\nsYInn3ySwYMHX1H+8OHDXH/99YSEmP/aMTExlClTBoAFCxYQFxdHo0aNuP/++zlz5gzvvPMOhw4d\nok2bNlfVblydP3+eTp06MX78eD+907zVsmVLypYt67bMlClTqFOnDvXr16dly5akpKTw2muv8cUX\nX9CgQQO++OILjh8/TocOHahduzZPPfUU2c2mntXvHkzNYciQITRq1IgpU6bQunVrBg4cSJMmTRgz\nZswV5xg2bBgPP/ww8fHxPPzww0yYMIF+/fplvH733XezZMmSq6796aefctttt9GgQQOeeeaZqxKW\nLwTF8N8SJUowdepUvvvuO+rWrRvocKwAGjgQNm1yX+a332DLFkhLg5AQqFcPSme3oCzQoAGMHu3b\nOMGMNhwyZAjr16+nTJkydOjQgRkzZvDaa6+RkJDAyJEjadKkyRXH9OjRgzvuuIPly5fTtm1bHnro\nIRo2bEhSUhJvvPEGCxcupHjx4vz9739n1KhRvPbaa4waNYrFixdTvnz5LOM4c+YMPXv25JFHHuGR\nRx7x7Zv05h/kWvnoH2T48OHMnz+fypUrc/LkSYoUKcLw4cNZt24dY8eOBaB///7ccccdvPbaa8ye\nPZsPP/zwqvO4+90DlCtXjg0bzGrA48aNIyUlJdu5Abdt28aKFSsoWrQoEyZM8Pgetm/fzhdffEFi\nYiLh4eH07duXiRMn+vzfMSgSyU033cSePXsYPnw4U6ZM8XyAFdR++80kETA/f/vNfSLxl7Vr19K6\ndWsqVDCTrz744IMsW7aMrl27ZntMTEwMP/74IwkJCSQkJNC2bVumTJnC+fPn2bZtG/Hx8QCkpKQQ\nFxfnVRxdunThpZde4sEHH8z9mypA4uPjeeyxx+jRowf33XdflmWWLVvGl19+CcDvfve7jNqfq9Wr\nV7v93T/wwANXlM/83NW9995L0aJFvX4PixYtYv369TRt2hQwNcvo6Givj/dWUCSSsLAwBgwYwOuv\nv25rJUHOmy+qq1ZB27aQkgJFisDEieDlZ26+EBERQadOnejUqRPXXXcdM2bMoEOHDrRv355JkyZd\n8/ni4+OZN28evXv39v0wan9U5Xxk3LhxrFmzhtmzZ9O4cWPWr1+fo/OoqtvfffHixd0+z+61sLAw\n0tK/8UCW94GoKo8++ihvvvnmtYZ9TYKmj2TgwIGUKlWKv/zlL4EOxcrn4uJg0SJ4/XXzM1BJ5Lbb\nbmPp0qUkJSWRmprKpEmTaNWqldtjNmzYkHEDblpaGlu2bKFq1ao0a9aMxMREdu7cCcDZs2fZsWMH\nACVLluT06dPZnnP48OGUKVOGP/7xjz56ZwXDrl27uP322xk+fDgVKlTgwIEDV/2uWrZsyWeffQbA\n3LlzOXHixFXncfe7z43Y2Fg2bdpEWloaBw4c4Ntvv72qTNu2bZk6dSpHjx4F4Ndff2XfPq9mhr8m\nQZNIypYty4ABA5g2bRpbtmwJdDhWPhcXB3/6k++SyLlz54iJicnYRo0aBcAbb7xxxX5X119/PSNG\njKBNmzbUr1+fxo0b06VLF7fXOXr0KPfccw916tShXr16hIWF0a9fPypUqMCECRPo1asX9erVIy4u\njh9++AGAp59+mo4dO7rtbB8zZgznz5/npZdeyuVvIn/o1asXcXFx/Pjjj8TExGTZtzF48OCMgQ7N\nmzenfv36tGnThm3btmV0tg8dOpRly5ZRu3ZtvvzyS6pUqXLVedz97nMjPj6eatWqUatWLfr370+j\nRo2uKlOrVi3eeOMNOnToQL169Wjfvr1fbtAOijXbmzRpouvWrePEiRPExsbSvn17pk6dGuiwrDyy\nfft2atasGegwLCtfy+r/iYisV9Um2RySIWhqJABlypRh4MCBtlZiWZblQ0GVSMD2lViWZfla0CWS\n9FrJl19+yebNmwMdjpVHgqEJ17JyKrf/P4IukYCplZQuXdre7R4kIiMjOX78uE0mlpWF9PVIIiMj\nc3yOoLiPJLP0Wslf/vIXNm/eTP369QMdkuVHMTExHDx4kGPHjgU6FMvKl9JXSMypoBq15erkyZPE\nxsbStm1bpk2bFqDILMuy8i87asuDqKiojL6STb6e68eyLCuIBG0iAdtXYlmW5Qt+TSQi0lFEfhSR\nnSLycjZleojINhHZKiKfuex/y9m3XUTeEaOkiGxy2ZJEJMeT9URFRTFo0CCmT59uayWWZVk55LdE\nIiKhwL+ATkAtoJeI1MpU5hbgT0C8qtYGBjr7mwPxQD2gDtAUaKWqp1W1QfoG7AO+zE2cAwYMsLUS\ny7KsXPBnjeQ2YKeq7lbVFOBzIPNEQX2Af6nqCQBVPersVyASKAJEAOHAL64Hikh1IBpYnpsgba3E\nsiwrd/yZSCoDB1yeH3T2uaoOVBeRRBFZLSIdAVR1FbAYOOxs81V1e6ZjewJfqBfDzg4dcr/SXXqt\nxN7tblmWde0C3dkeBtwCtAZ6AeNFJEpEbgZqAjGY5HOniLTIdGxPINvFFUTkaRFZJyLrDh8260tk\nl0zSayUzZsxg48aNuX5TlmVZwcSfieRn4AaX5zHOPlcHgZmqelFV9wA7MImlG7BaVc+o6hlgLpAx\nobeI1AfCVDXblWZU9X1VbZI+BjolBbJYzjjDgAEDiIqKsn0llmVZ18ifiWQtcIuIVBORIpgaxMxM\nZWZgaiOISHlMU9duYD/QSkTCRCQcaAW4Nm31wk1tJCshIdC6dfav21qJZVlWzvgtkajqJaAfMB+T\nBCar6lYRGS4i9zrF5gPHRWQbpk9ksKoeB6YCu4DvgM3AZlX92uX0PbiGRFKkCJQrB86yxdnq378/\nUVFRtq/EsizrGgTFFCk339xEd+1ax8SJ0Lu3+7LDhw9n6NChbNiwgYYNG+ZNgJZlWfmQt1OkBEUi\nadKkiZ4/v47wcNi4EUSyL/vbb78RGxtLq1atmDFjRt4FaVmWlc/YubYyGTwYNm+G+fPdlytdujTP\nP/88X331le0rsSzL8kLQJJLevSEmBkaM8FzW9pVYlmV5L2gSSZEi8PzzsHQprFnjvqxrrWTDhg15\nE6BlWVYBFTSJBKBPHyhTBv7+d89lba3EsizLO0GVSEqUgH79YMYM+OEH92XTayUzZ860tRLLsiw3\ngiqRADz3HERGwj/+4bls//79KVOmjK2VWJZluRF0iaRCBXjiCfjkE/g584QtmdhaiWVZlmdBl0gA\nXngB0tLgn//0XPa5556jTJkyDBs2zO9xWZZlFURBmUiqVYMHHoD33oMTJ9yXTa+VfP3116xfn+0c\nkZZlWUErKBMJwEsvwZkz8J//eC5r+0osy7KyF7SJpH596NgRRo+G8+fdly1VqhQvvPCCrZVYlmVl\nIWgTCcDLL8OxYzBhgueytq/Esiwra0GdSFq2hNtvh5Ej4dIl92XTayWzZs1i3bp1eROgZVlWARDU\niUQEhgyB3bth2jTP5Z977jnKli1r+0osy7JcBHUiAejSBWrUMJM5eppR39ZKLMuyrhYUieTChQvZ\nvhYSYkZwbdoE33zj+Vz9+vWztRLLsiwXQZFI9uzZ4/b1Bx+ESpW8m8zRtVaydu1aH0VoWZZVcAVF\nIjl79qzbD/2ICDPFfEICeJMbbK3EsizrsqBIJCEhIYwZM8Ztmaefhqioa6uVzJ4929ZKLMsKekGx\nZvt1112nJ06cYO/evVSqVCnbcq+8Am++aaaYr17d/TlPnz5NbGwscXFxzJo1y8cRW5ZlBZ5ds91F\ndHQ0ly5d4uuvv3Zbrn9/s5KiN1PMlyxZkhdffNHWSizLCnpBkUgiIiL46aefeOaZZ9yWu+46M8X8\n//4Hhw97Pm96X4m9292yrGAWFIkE4KabbgIgNTXVbbkXXzR3uY8e7fmc6bWSOXPm8O233/oiTMuy\nrALnmhKJiISISKlrKN9RRH4UkZ0i8nI2ZXqIyDYR2Soin7nsf8vZt11E3hERcfYXEZH3RWSHiPwg\nIr/3Np7hw4cTFxeHu36hG2+E++83swKfPOn5nP369aNcuXJ2BJdlWUHLYyIRkc9EpJSIFAe+B7aJ\nyGAvjgsF/gV0AmoBvUSkVqYytwB/AuJVtTYw0NnfHIgH6gF1gKZAK+ewV4CjqlrdOe9Sb94oQExM\nDGvXrmXJkiVuyw0ZAqdPw7hxns9payWWZQU7b2oktVT1FNAVmAtUAx724rjbgJ2qultVU4DPgS6Z\nyvQB/qWqJwBU9aizX4FIoAgQAYQDvzivPQG86ZRPU9UkL2IBoHfv3pQvX97jUOCGDaFDB9O8lZzs\n+bx//OMfKVeunO0rsSwrKHmTSMJFJByTSGaq6kXMB70nlYEDLs8POvtcVQeqi0iiiKwWkY4AqroK\nWAwcdrb5qrpdRKKc414XkQ0iMkVErvMiFgAiIyN59tlnmTlzJrt27XJbdsgQ+OUX+Phjz+dNr5XM\nnTuXNWvWeBuOZVlWoeBNInkP2AsUB5aJSFXglI+uHwbcArQGegHjRSRKRG4GagIxmORzp4i0cMrH\nACtVtRGwChiZ1YlF5GkRWSci644dO5ax/w9/+AOhoaGMHTvWbWBt2kDTpmaKeQ/988DlWontK7Es\nK9h4TCSq+o6qVlbVzmrsA9p4ce6fgRtcnsc4+1wdxKnlqOoeYAcmsXQDVqvqGVU9g2lSiwOOA+eA\nL53jpwCNson7fVVtoqpNKlSokLG/UqVKfPTRR/Tr189t8OlTzO/cCV9+6bYoYGsllmUFL2862wc4\nne0iIh+KyAbgTi/OvRa4RUSqiUgRoCcwM1OZGZjaCCJSHtPUtRvYD7QSkTCnWa0VsF3NcKuv048B\n2gLbvIjlCg899FDGcGB3unaFW27xbop5sCO4LMsKTt40bT3hdLZ3AMpgOtpHeDpIVS8B/YD5wHZg\nsqpuFZHhInKvU2w+cFxEtmH6RAar6nFgKrAL+A7YDGxW1fTb0ocAw0RkixPLC9691SutXbuWp59+\n2u19JaGhZor5DRtg0SLP5yxRogSDBw9m7ty5rF69OidhWZZlFTge59oSkS2qWk9ExgBLVHW6iGxU\n1YZ5E2LuNWnSRDMvRDV58mQeeOABZs6cyT333JPtsRcuQLVqULu2d+uVnDlzhtjYWJo2bcrcuXNz\nG7plWVbA+HKurfUisgDoDMwXkZJAWm4DDLT77ruPG264gdEebmGPiICBA2HhQli/3vN502sl8+bN\ns7USy7KCgjeJ5EngZaCpqp7D3NvxuF+jygNhYWH069ePhIQEtmzZ4rbsM89AqVLeTTEPZgRX+fLl\nbV+JZVlBwZtRW2mYEVd/FpGRQHNVdf/JW0A89dRTFC1a1OMNiqVLQ9++MG2aGcXliWutZNWqVT6K\n1rIsK3/yZtTWCGAAZnTUNqC/iPzN34HlhbJly/LCCy9Qo0YNj2UHDIDwcHNfiTf69u1rayWWZQUF\nrzrbgQZOzSR9Dq2NqlovD+Lziaw623PimWfMne5790LFip7Lv/XWWwwZMoSVK1cSFxeX6+tblmXl\nJV8vbBXl8rh0zkLKvy5dusS0adNISUlxW27wYLh4ETy0hGVwrZXYebgsyyqsvEkkbwIbRWSCiHwM\nrAf+6t+w8lZCQgLdu3dnypQpbsvdfDP8/vfw73/Db795Pm96X8n8+fNtE5dlWYWWV2u2i8j1mKnc\nAb4FQlT1kD8D8yVPTVtpaWnUqlWLkiVL8u233+IsfZKl9euhSRMzguullzxf++zZs8TGxpKUlERK\nSgrh4eE5eQuWZVl5zqdNW6p6WFVnOtsRoFDdIBESEsKAAQNYt26dx1FWjRtDu3ZmivkLF9yfd9iw\nYZQoUYKkJDPTfZEiRRAR28xlWVahktOldrP/yl5APfLII0RFRXkcCgxmMsfDh+GTT9yXGzZsGKqa\nsSKjiBAfH8+AAQN8EbJlWVa+kNNE4s16JAVK8eLF6dOnD99//z0XPFQ12raFRo3grbe8m2I+3eTJ\nk1m7di0tW7bk0KEC0zJoWZblVrZ9JCLyLlknDAEeVVWv124PNG+H/547d47IyEhCQjzn1ylToEcP\nmDrVdMB7MmzYMIYNG8aiRYvo2rUr5cqVY8GCBVSvXt2bt2BZlpXnvO0jcZdIHnV3oKp6sXZg/nCt\n95GcO3eOkJAQIiMjsy2Tmgq33gplysCaNWb9Em+tW7eOTp06ISLMnTuXxo0be3+wZVlWHsl1Z7uq\nfuxu8224+ceBAweIiYnhYw9r7IaGwosvwtq1sHjxtV2jSZMmJCYmUqxYMdq0aUNCQkIuIrYsywos\nb+5s/5qrm7h+A9YB76lqsp9i85lrqZGoKk2bNuXs2bNs3brVbTNXcjLExkL9+jB//rXH9fPPP3PX\nXXfx008/8dlnn/F7b9rILMuy8ogvh//uBs4A453tFHAas5rh+NwEmR+JCAMGDOCHH37gGw8LkERG\nminmFyyAjRuv/VqVK1dm2bJlNGnShPvvv5/33nsvh1FblmUFjjc1krWq2jSrfSKyVVVr+zVCH7jW\nPpILFy4QGxtLw4YNmTNnjtuyJ09ClSrQuTN8/nnO4jt37hw9evRg9uzZvP7667zyyitub4q0LMvK\nC76skZQQkSouJ64ClHCeup+cqoCKiIigb9++zJ07l927d7stGxUFzz5rRnHt2pWz6xUrVozp06fz\n8MMP8+qrrzJgwADS0gr82mGWZQUJbxLJC8AKEVksIkuA5cCLIlIcKLSd7n/4wx9YtWoVN954o8ey\nAwdCWBi8/XbOrxceHs6ECRMYNGgQ7777Lg899JDHSSQty7LyA2/n2ooAbnWe/lgQOthd+WoaeXf6\n9IFPPzVTzF93Xc7Po6q89dZbvPzyy9x1111MmzaN4sWL+yxOy7Isb/msaUtEwoFngFed7SlnX6F3\n8eJF+vTpw7vvvuux7ODBZu6td97J3TVFhCFDhvDBBx/wzTff0LZtW44fP567k1qWZfmRN01b/wEa\nA/92tsbOvkIvPDycnTt3MnLkSC5duuS2bPXqcN99Zor506dzf+0nn3ySadOmsWnTJlq0aMGBAwdy\nf1LLsiw/8CaRNFXVR1U1wdke5/KU8oXegAED2L9/P1999ZXHskOGmFFc77/vm2t37dqV+fPn8/PP\nP9O8eXO2b9/umxNblmX5kDcWT3xxAAAgAElEQVSJJFVEbkp/IiI3AtcwVWHBds8991CtWjVGjx7t\nsWzTptCmDYwa5XmKeW+1atWKpUuXcvHiRe644w7WrFnjmxNblmX5iDeJZDCwWESWiMhSIAEzkssj\nEekoIj+KyE4ReTmbMj1EZJuIbBWRz1z2v+Xs2y4i74hzY4UTx48issnZor2JJadCQ0N57rnnWLFi\nBevXr/dY/uWX4dAhmDjRdzE0aNCAxMREoqKiaNu2LQsWLPDdyS3LsnIrfb0MdxsQAdRztgjgdi+O\nCQV2ATcCRYDNQK1MZW4BNgJlnOfRzs/mQKJzjlBgFdDaeW0J0MSbuNO3xo0ba26cPHlS+/fvr3v3\n7vVYNi1NtWFD1Ro1VFNTc3XZqxw6dEjr16+v4eHhOmnSJN+e3LIsKxNgnXrxGevtCokXVHWLs10A\n3C9ubtwG7FTV3aqaAnwOdMlUpg/wL1U94VznaPolgUgnAUUA4cAv3sTqD6VLl2bMmDFUrVrVY1kR\nswTvjz+CF90q1+T6669nyZIlxMXF0bt3b8aOHevbC1iWZeWAP1dIrAy4DjU66OxzVR2oLiKJIrJa\nRDoCqOoqYDFw2Nnmq6prT/NHTrPWq+lNXlcFKPK0iKwTkXXHjh3z8m25t2LFCiZPnuyxXPfucOON\nZl13L27TuSZRUVHMmzePe++9l+eee46hQ4dmrMBoWZYVCIFeITEM07zVGugFjBeRKBG5GagJxGCS\nz50i0sI55kFVrQu0cLaHswxQ9X1VbaKqTSpUqOCTYP/+97/Tr18/kpPd348ZFmammF+zBpYt88ml\nr1C0aFGmTp3KE088wfDhw+nbty+p17JUo2VZlg9lm0hE5GsRmZnF9jVQzotz/wzc4PI8xtnn6iAw\nU1UvquoeYAcmsXQDVqvqGVU9A8wF4gBU9Wfn52ngM0wTWp4YOHAgx44dY9KkSR7LPvYYREfDiBH+\niSUsLIwPPviAIUOGMG7cOHr27OlxiWDLsix/cFcjGQm8ncU2EujsxbnXAreISDURKQL0BGZmKjMD\nUxtBRMpjmrp2A/uBViIS5txF3wrY7jwv75QPB+4GvvciFp+48847qVOnDmPGjPHYnFS0KPTvD/Pm\nwebN/olHRBgxYgQjR45k6tSpdO7cmdO+uBvSsizrWnjTI5/TDZNwdmBGb73i7BsO3Os8FmAUsA34\nDuipl0d8vQdsd14b5ewvDqwHtgBbgTFAqKc4cjtqy9X48eMV0MWLF3ss++uvqiVKqPbu7bPLZ+vj\njz/W0NBQbdy4sR49etT/F7Qsq9DDy1FbXk3aWND5ctLG8+fP07hxY1577TV69uzpsfyLL5obFF94\nwUyhEhfnkzCyNHv2bO6//35uuOEG5s+fT2xsrP8uZllWoeftpI02keSAqnq98NTMmdClixkWHBkJ\nixb5N5kkJiZy9913U6xYMebPn0+dOnX8dzHLsgo1Xy5sZWUiIqSmprJ161aPZbduNUlE1UybsmSJ\nf2OLj49n+fLlALRo0YLExET/XtCyrKCXk1FbM0Ukc6d50Hn++eeJi4vj1KlTbsu1bm1qIgBpabB/\nv+/vLcmsTp06JCYmUqFCBdq3b8/s2bP9e0HLsoKaN6O29gDngfHOdgbTeR7UHnzwQU6fPs2ECRPc\nlouLM81Zr78O7dvDuHGmv8TfK+nGxsayYsUKatasSZcuXfjkk08AGDZsmH8vbFlW0PHYRyIi6zK3\nkWW1Lz/z1wqJzZs35+jRo+zYsYOQEM+thGlpZlned9+Fhx+GDz+EcD8vEXbq1Cm6detGQkICo0aN\n4vnnn7d3wluW5RVf9pEUd6aOTz9xNcww3KA3YMAAdu3a5XXTUUgIjBkDw4fDJ59At25w7px/YyxV\nqhRz5syhe/fuPP/88wC8/vrrfPfddzahWJblE94kkkHAEpdp5BcDA/0bVsFw3333ERMT49X8W+lE\n4NVX4T//gTlzTHPXiRN+DBJ48803mTp1asbz1157jXr16lGuXDlefPFFEhMT7RQrlmXlmFfDf0Uk\nArjVefqDmhmACwx/NW0B7Nq1i9jYWEJDQ6/52ClT4MEHoUYNmD8fKlXyQ4CZiAiHDx9m5syZzJgx\ng4ULF3Lx4kWio6Pp0qULXbt2pW3btkRERPg/GMuy8jWf3UciIsWA54GqqtpHRG4BaqjqLN+E6n/+\nTCTp0tLSvOonyWzhQujaFSpUgAUL4JZb/BCcCxG5oknr1KlTzJkzhxkzZjBnzhxOnz5NyZIl6dy5\nM127dqVz586UKlXKv0FZlpUv+bKP5CMgBWfSRMzEi2/kIrZCZ9asWVSrVo2cTFffrh0sXgxnzkB8\nPGzY4IcAXQwdOvSK56VKlaJnz558/vnnHDt2jDlz5tCzZ08WL15Mr169KF++PJ06deL999/nyJEj\n/g3OsqwCyetRWyKyUVUbOvs2q2r9PInQB/xdI9m2bRu1a9fmjTfe4JVXXsnROX78ETp0MP0lX31l\n1n4PpNTUVFavXs2MGTOYPn06u3btQkSIi4uja9eudOvWjZtvvjmwQVqW5Ve+bNpaCbQFElW1kYjc\nBExS1Tybvj238qJp66677uL7779nz549FClSJEfnOHgQ7roLdu6Ezz83o7ryA1Vl69atTJ8+nRkz\nZrDBqTbVrl2bbt260bVrVxo1auT1tDGWZRUMvmzaGgbMA24QkYnAImBI7sIrfAYMGMChQ4euGB11\nrWJizEJYjRqZVRY/+MCHAeaCiFCnTh1effVV1q9fz969exk9ejQVKlTgb3/7G02aNKFq1ar079+f\nxYsXc+nSpUCHbFlWHvJ21FY5oBlm2vfVqprk78B8Ka8622vWrElUVBRr1qzJ1bnOnjWJZN48+Nvf\n4OWXzbDh/CgpKYlZs2Yxffp0FixYQHJyMmXLluWee+6ha9eudOjQgWLFil1xzLBhw+wd9pZVAPiy\naWuRqrb1tC8/y4tEAjBv3jyKFStGixYtct3Mk5JiVlmcNAkGDYKRI80NjfnZ2bNnmT9/PtOnT2fW\nrFmcPHmSokWL0rFjR7p27crdd99N2bJlrxo5ZllW/uRtIglzc4JIoBhQXkTKYGojAKUw66hbmXTs\n2NFn5ypSBD79FMqXh3/+E5KS8mZKldwoXrw49913H/fddx8XL15k6dKlzJgxI6PDPjQ0lFatWgFm\ngELNmjVtv4plFQLuvuM+g1mN8FbnZ/r2FTDW/6EVTD///DP9+vXjwIEDuT5X+pQqr7+ed1Oq+Ep4\neDjt2rVj7Nix7N+/n6eeeorU1FQSEhIA01EfEhJCvXr1+Pjjjzl48GCAI7YsK6e8adp6TlXfzaN4\n/CKvmrYA9u3bx4033sjgwYMZMWKEz847bhz07WtmE541C8qU8dmp85yI8MEHH7Bw4UIWLVqUcf9N\njRo1aNeuHW3btqVNmzZERUUFOFLLCm4+XSFRROoAtYDI9H2q+r9cRZiH8jKRAHTv3p2EhAQOHjx4\nVUdzbkydaqZUqV4976ZU8QfXPpK0tDS+//77jKSydOlSzp49S0hICE2aNMlILM2bNycyMtLDmS3L\n8iVfdrYPBVpjEskcoBOwQlW7+yDOPJHXiWT58uW0bNmScePG8cwzz/j03IsWmSlVypfPmylV/MHd\nqK2UlBTWrFnDokWLWLhwIatXryY1NZXIyEhatGhB27ZtadeuHQ0aNMjR/GaWZXnPl4nkO6A+sFFV\n64vIdcCnqtreN6H6X14nElWlSZMmnD9/nq1bt/q8Q3ndOujUyQwJnjfP3HdSWJ06dYply5ZlJJbv\nv/8egLJly9KmTRvatWtHu3btuOmmm2zHvWX5mC8TybeqepuIrAfaAKeB7ap6q9sD85G8TiQAn3/+\nOYsWLeKf//wnJUqU8Pn589uUKnnlyJEjJCQksHDhQhYuXJgxqKFKlSoZSeXOO+/kuuuuC3CkllXw\n+TKR/Bv4P6An8AJmqd1Nqvq4LwLNC4FIJHnh559NMtm509xvct99gY4ob6kqO3fuzOhfSUhI4ISz\nuEvdunUzEkvLli2zTOb2xkjLcs+nne0uJ40FSqnqlpyHlvcCmUjWrFlDxYoVqVq1ql/O/+uv8Lvf\nwbffmpFdffr45TIFQmpqKhs3bsxoBlu+fDkXLlwgLCyMZs2aZXTc33777YSHh9sbIy3Lg1wnEhFx\n2/Kuqh4nPBeRjsAYIBT4QFWvGg8rIj0w83kpsFlVezv73wJ+h7nX5RtggLoEKyIzgRtVtY6nOAKV\nSI4fP06lSpXo06cPY8f679abgjSlSl46f/48K1euzEgs69evJy0tjRIlStCiRQvmzp3LsGHDqFCh\nwhVbdHQ0ZcuWtZ35VtDzRSJZ7DyMBJoAmzF3t9cD1qlqXJYHXj4+FNgBtAcOAmuBXqq6zaXMLcBk\n4E5VPSEi0ap6VESaA/8AWjpFVwB/UtUlznH3Ad2Bevk5kQA8/vjjTJ48mYMHD1LGjzd/XLxoplT5\n7LOCM6VKXjtx4gTPPvusV0sjiwjlypW7Ksm4JhvX5+XLlycsLNuJIrJlm9es/MyXfSRfAkNV9Tvn\neR1gmKfhvyIS55S7y3n+JwBVfdOlzFvADlX9IItjxwJ3YJLXMuBhVd0uIiUwsxE/DUzO74lk06ZN\nNGzYkH/84x+8+OKLfr1WWppJIu+8Aw8/nP+nVAk0ESElJYXjx49z7Nixq7ajR49ete/48ePZNoeV\nKVPGq6STvhUpUsQ2r1n5Wq7n2nJRIz2JAKjq9yJS04vjKgOu84QcBG7PVKa6E2wipvlrmKrOU9VV\nTo3oMCaRjFXV7c4xrwNvA24nCxGRpzHJhipVqngRrn80aNCAli1b8vbbb9OlSxdu8eONHyEhMHo0\nREfDn/9s+k8mTwYf3hNZ6ISHh1OxYkUqVqzoVfnU1FR+/fVXjwln586drFq1iqSkJFJTU7M8V/oS\nxs8++yzNmzenefPmdhizVSB5k0i2iMgHwKfO8wcBX3W2hwG3YG54jAGWiUhdoDxQ09kH8I2ItMAM\nPb5JVQc5Hf/ZUtX3gffB1Eh8FG+OvPPOO3Tp0oWkpCS/JhIwfSOvvGJuWPzDH6B9+4I/pYq/ZF52\n2BuhoaEZNQpvpKWlcfLkySuSzIcffsjs2bM5deoUAO+99x7vvfceABUqVCAuLo7mzZsTFxdHkyZN\nfDo7gmX5gzdNW5HAH7jcX7EM+I+qJns4zpumrXHAGlX9yHm+CHgZk1giVfV1Z/9rQDImkbyKWUM+\nDIgGVqpqa3ex5IfhvykpKRkrJ+7fvz9PakmFZUqVwkxESE1NZdu2baxatYqVK1eycuVKduzYAUBY\nWBgNGjTIqLHExcVxww032FqLlSf8Mvz3GgMIw3S2twV+xnS291bVrS5lOmI64B8VkfLARqAB0A7o\nA3TENG3NA0ar6tcux8YCs/J7H0lmkyZN4vHHH+fzzz+na9eufr9e+pQq5crBW2/Brl3QurWZ/NEK\nvOz6SJKSkli9ejUrV65k1apVfPvtt5xzpn6uXLlyRq2lefPmNGzYMMfLO1uWO94mElQ1yw3TkQ3w\nHaYp64otu+MynaMzJpnsAl5x9g0H7nUeCzAK2OZcp6ezPxR4D9juvDYqi3PHAt97E0fjxo01v0hK\nStJmzZppSEiIfvDBB3lyzbVrVUuXVgVVEdWiRVVXrsyTS1seDB061KtyKSkpum7dOn333Xe1V69e\nGhsbq5gh8xoREaHx8fE6ePBg/fLLL/Xw4cP+DdoKGpgRuh4/Y90N/71eVQ+LSJZ30qnqPo9ZKp/I\nTzUSMCsJ/v73v2f+/PmMGDGCl156ye9NFS+8AKNGXX7eqBG8/Ta0bGmHCRdUhw4dYtWqVRlNYuvX\nryclJQWAatWqXdEcVrduXbfDk+0wZCsrInJYVT02ivutaSs/yW+JBEyfyWOPPcakSZNYvXo1t9+e\neUCbb61aBW3bwoULpkM+IsIsklWtGjz6KDzyiHlsFVwXLlxgw4YNGf0sK1eu5MiRI4BZvfK2227L\nSC7NmjWjbNmyGcfaYciX5YekmhcxqCqXLl3i/PnzJCcnk5ycfMXj5ORkWrdujap6/JbrrkZyGlN1\nvuolE4OWyt3byDv5MZGAGdGzaNEi2rfPm4mUV62CJUtMH0n9+jBjBnz0kelHUTUTPz72GPz+91C8\neJ6EZPmRqrJv376MfpaVK1eyefPmjOHIt956K3Fxcdx+++08++yzzJ8/n+LFi2e55eRmy2uVHz7A\nIe+SalpaGikpKVluNWrU4Ntvv73qw93Xj9PS0jzGmatEUpjk10Tiavny5fzrX//io48+omjRonl6\n7f374X//gwkTTGd8iRLQo4dJKnfcYadbKUzOnj3L2rVrWblyJRMmTOCnn37y6rgiRYpkmWCKFSuW\nbfLxdouIiEBEsvwAV1XS0tJITU3l0qVL2f7M6WtZlXnqqacYM2ZMth/y6dvFixc9lnFXPrv7i65V\nREQEkZGRGVvRokU9Pnb32syZM6+Y/cGniUREorlyhcT9OXjPAdGkZEld17hxoMNw69Dhw+zYsYPS\npUtTt06dPPkGmJkCp36Dw0fg2FFITYOikVCxIlxXESIj8jwky88USE5OZs2aNTRs0IBU50M7LTWV\n1NTUHD335ltuZqGhoaSmppr5zVw7cn3/lnNERAhxkp2EhJjHPv7566+/cvzXX6+6dsWKFalcqRIh\noaGEhIRctfnze54sXepVIvH4aSUi92LuJK8EHAWqYkZT1c5tkNZlla6/nrDQULb/8AObNm2iXr16\neT6kU4DSpc12y81wLAmOHIE9e81WJsoklfIVINR20BcKAhR1ljAuXbq0T86pYBKLF0knKSmJkydP\nZnw7T/9ZulQpSkdFZdRURASBK57j+poPy6xevZr4+PiMD/n0Y/ytcuXKGY+XLF1K61at8uCqvuHN\nDYmbgTuBharaUETaAA+p6pN5EaAvFISmrXQLFiygW7duVKxYkcTERK+n7vCnvXvh44/NtmcPlCoF\nDzxgmr7i4mzTV2GQH/on8kuHf36IIz/E4MTh1agtb75XXlTV40CIiISo6mLMbMCWH3To0IGEhATa\nt2/v9TQc/hYbC0OHmgW0liyBbt1g4kSIj4caNczU9QcPBjpKKzcCnUTyk5xMnVMYY3Ac8qaQNzWS\nhUBX4E3MHFhHgaaq2jy3EeaVglQjyezAgQPs37+f+Pj4QIdyhdOnYdo0M+pr2TJTK2nfHh5/HLp0\ngTweL2AVAvmhVmRdyZfTyBfHzHMlmAkbSwMTnVpKgVCQE0m3bt2YN28ekydP5p577gl0OFnatety\n09f+/aaPpWdPk1Ruu802fVlWQeWLha3+BXymqom+Di6vFeREcuzYMTp37szGjRv58MMPefTRRwMd\nUrbS0kzT10cfmdrK+fNw662mL+Xhh+2kkZZV0PgikQwAegLXY1YxnKSqG30aZR4pyIkE4PTp03Tr\n1o1FixYxcuRIXnjhhUCH5NGpUzBlikkqiYlmGpa77jL9Kmlp0K6dnTjSsvI7XzZtVcUklJ5AUWAS\nJqns8EWgeaGgJxIw01889NBD/PTTT6xevZrIyEjPB+UTP/1kbnYcPx6OHTP7QkPh3/+GPn1s05dl\n5Vd+mUZeRBoC/8WslR6ai/jyVGFIJGDG2P/222+ULVuW5ORkwsPDzQ1cBcRf/wqvvWZqJOluvtnM\n8/XQQ3auL8vKb7xNJB6H/4pImIjcIyITgbnAj8B9PojRukahoaGULVuWtLQ0evToQY8ePUhOdru+\nWL5y551mssjQUDOq65VX4IYbTHK58UZo1cqsM//bb4GO1LKsa5FtIhGR9iLyX8xa632A2Zhlbnuq\n6ld5FaB1tZCQENq2bcuXX35J586dM5Zsze/i4swEka+/bn6+8QYkJJgbHv/6V/jlF3jqKXP3fK9e\nMHcuXLoU6Kgty/LEXWd7AvAZME1VT+RpVD5WWJq2Mvv00095/PHHqVevHnPnziU6OjrQIeWKKqxd\nayaQnDQJfv0VrrvOLBf8yCNmxmLLsvJOwJfazU8KayIBmDNnDt27d6dhw4asWLGi0KzlnZICc+aY\npDJrFly8CPXqmYTSuzdcf32gI7Ssws8mEheFOZEAJCYmUqRIEZo2bRroUPzi+HH44gv45BNYvdoM\nJW7f3iSVrl2hWLFAR2hZhZNNJC4KeyJxNWLECFq2bEnz5gVmBptrsmOHSSj/+5+5i75kSeje3SQV\nu2ywZflWrkdtiUgVN6+1yGlglv+cPn2a//73v7Rr1465c+cGOhy/qF7ddNbv2WPuor//fpg61azu\nWK0a/PnP8OOPgY7SsoKLu+9vS0TkJRHJuFFBRK4TkU+Bf/o/NOtalSxZkhUrVnDrrbdy7733MnHi\nxECH5DchIZeHCx85Ap99BrVqwZtvmmlZmjUzNzweLzAzwllWweUukTQGbgI2icidzpQp3wKrgNvy\nIjjr2kVHR7NkyRLi4+N56KGHGDt2bKBD8rtixS4PFz54EEaOhHPn4I9/NJ3y991n1qdPSQl0pJZV\nOHkzRcoATA3kENBMVQvcyhPB1EeSLjk5mQcffJBu3brx0EMPBTqcgNi82fSlTJxo7lEpV87MSvzI\nI+b+lKVLoXVrO+eXZWXHF5M2RgF/B24HXgI6A22BAaqa4MNY/S4YEwmAqmYMB96wYQP169cvUFOq\n+MqlS/DNNyapzJgByclmfi9VCAszNZeWLc1d9jfcANHRttPessA3iWQ38G9gtKpecvY1cPbtU9Ve\nXgTRERgDhAIfqOqILMr0AIZhlnrerKq9nf1vAb/DNL99g0lgKiLzMDMShwHLgT+qaqq7OII1kaTb\ntWsXtWrVokuXLkyYMIFiQTxe9rff4Ikn4Msvsy8THg6VK19OLK5bTIz5Wb68nWzSKvx8kUhismvG\nEpE+qjreQwChwA6gPWaalbVAL1Xd5lLmFswU9Xeq6gkRiVbVoyLSHPgH0NIpugL4k6ouEZFSqnpK\nzFftqcAUVf3cXSzBnkgARo4cyeDBgylXrhzPPPMMffv2pXLlyoEOKyBWrYK2bU2fSZEiZtTX9dfD\ngQNXbwcPmu3ixSvPERl5Oam4JhjXLSrKJhurYPM2kYRl94K7vhBPScRxG7BTVXc7AX0OdAG2uZTp\nA/wrfQoWVT2afgkgEiiCWZkxHPjFKZM+sVSY83rhvxHGB1588UXi4uJ4++23efPNNxk7dixHjhyh\naBCuiZs+59eSJVf2kTRsmHX5tDQ4evTqBJP+ePFiOHQIUjPVi4sXzz7J3HCDOWbtWttPYxV82SYS\nH6gMHHB5fhDT3+KqOoCIJGKav4ap6jxVXSUii4HDmEQyVlW3px8kIvMxiWouplZyFRF5GngaoEqV\nbG+JCSrx8fHEx8eze/du1q9fT9GiRVFVnn32Wdq3b0/Xrl0JC/Pnn0T+ERfn/Yd3SIiZSLJiRchu\n8oDUVDh8+MoE47rNm2eGKWfVABAaamZA7tvXNJlZVkHjtzvbRaQ70FFVn3KePwzcrqr9XMrMAi4C\nPYAYYBlQFyiP6Vt5wCn6DfCSqi53OTYSmAiMU9Vv3MVim7ayd/ToUZo1a8aePXuoWrUqzz33HE8+\n+SRRUVGBDq3QuXjR1EIOHICxY2Hy5CsTiwg0bmxWkrzrLnMvTHh44OK1LJ+tR5ILPwM3uDyPcfa5\nOgjMVNWLqroH06dyC9ANWK2qZ1T1DKbmccX3R1VNBr7CNJdZORQdHc1PP/3E9OnTiY2N5cUXXyQm\nJobExMRAh1bohIdD1apwxx0wYIDpZ0lfm2X8eBg2zKzXMmKEGUVWrpyZS+w//4HduwMdvWVlz581\nkjBMYmiLSSBrgd6qutWlTEdMB/yjIlIe2Ag0ANph+k86Ypq25gGjgcVASVU97Jx/IrBcVd3edWdr\nJN7buHEj77//Pm+//TbFihVj9uzZREZGcueddxaamYXzi1Wrru6nATh50qzTMn++2fbtM/tvvhk6\ndDC1lTZtzDxjluVP+WLSRhHpjEkAocB/VfWvIjIcWKeqM52RV29jEkYq8FdV/dwZ8fVvzKgtBeap\n6vMich0wC4jA1KYWA4PShydnxyaSnIuPj2flypXUrVuXgQMH0rt37wK1XnxBp2rWvE9PKosXm7v2\nw8OheXOTVDp0MAMF7L0vlq/li0SSX9hEknPJyclMmjSJ0aNHs2XLFipUqMBbb73FY489FujQgtKF\nC7By5eXEsmmT2V+hgpla/667zE+7XovlC/mhj8QqBCIjI3n88cfZtGkTCQkJxMXFUapUKQCOHTvG\nxo0bAxxhcImIMM1aI0bAxo1mpNj//mcSyMKF8OijUKmSWU3ypZfMMOfk5EBHbRVEq1YBVK7oTVlb\nI7FybPjw4QwdOpSWLVsyaNAg7rnnnqCcgiW/SEsz84stWGBqKytWmJFiRYuafpj00WA1atgbJYNJ\naiqcPn1t2969sGwZpKU1QXWdx78Wm0isHDt58iQffvgh7777Lvv27aNatWoMHDiQ5557znbM5wNn\nzpjO/PTEsmOH2V+lyuVO+7Zt4Ycfsu70twIjMdHMDdeokUn6p0+bf8trTQbp2/nz3l03NNQM4ChZ\n0jShHj0KYBNJBptI/OvSpUvMmDGD0aNHU758eWbMmAGYe1Sio6MDHJ2Vbu/ey30rixbBqVOmZiJi\najOhodCpE9x4o5maP30rXty7x0WL5rzDP7sRbHktcxyq5kP1/Hn3W3Ky5zLelDt71kwy6q0SJS5/\n+Od2i4y8XFNNn0bo/PnGqrre47+qTSSWT50/f56iRYvy008/UatWLe6++24GDRpEixYtbC0lH7l0\nCdasgVdfNSPB0pUsaRLK2bNXzy/mjcjIa0s+xYqZb77/+Y+JKSwM+vc399tcumS21FTfP87qtdOn\nzc2i6R+JRYqY30FOPyJDQkxy9WaLjDQ/N2+G5cvNNUNCzDLSDz6Y9Qd/8eL+Ham3ahU0bx7zs+rB\nGE9lbSKx/OLYsWOMGRKC4F8AABUvSURBVDOGcePGcfz4cRo2bMjAgQN54IEHiIiICHR4liPzBJaL\nFl2uEVy8ePlb8rlzlzfX59k99qbc2bNXz0/mSUiISTahoeanLx/v2AHffWeuI2JuHG3d2vOHf3Zb\nTmYlcPfvEQh2+K8Lm0gC59y5c0ycOJHRo0eza9cu9u/fT3R09BVrpViBFchmpYsXTY2oSxfzODzc\nzMbcrFnWH/r+/JPJLx/i+aWZD2wiuYJNJIGnqmzfvp1atWoB0LlzZ2rWrMnzzz8ftNPZW5fllw/P\n/BJHfmETiQubSPKX5ORk+vTpw6RJkwgJCeGRRx5h8ODB1KhRI9ChWZblwt6QaOVbkZGRfPLJJ+zc\nuZOnn36aiRMnUrNmTWbNmhXo0CzLygGbSKyAiY2NZezYsezbt4+hQ4fSpk0bAObNm8eiRYsIhtqy\nZRUGNpFYARcdHc3QoUMpXrw4ACNGjKBdu3bcdtttTJs2jdRrHdpjWVaesonEynfmz5/P+++/z8mT\nJ+nevTu1a9fm66+/DnRYlmVlwyYSK9+JiIigT58+/PDDD0yePJlixYpx3pnn4cyZM5w5cybAEVqW\n5comEivfCg0N5f7772f9+vV0794dgDFjxlClShWGDh1KUlJSgCO0LAtsIrEKABEhxJkLokOHDrRq\n1Yrhw4dTpUoVBgwYwP79+wMcoWUFN5tIrAKladOmTJ8+na1bt/LAAw/w73//m759+wY6LMsKajaR\nWAVSrVq1+Oijj9i1axdvv/02AHv27KFbt26sXr06wNFZVnCxicQq0KpUqZJxR/z27dtZunQpcXFx\ntG7dmnnz5tl7USwrD9hEYhUanTt3Zv/+/YwaNYqdO3fSqVMnmjVrxqVrWeDBsqxrZhOJVaiUKFGC\nQYMGsXv3bv773//SuXNnwsLCAJg5cybJdgFzy/K5sEAHYFn+UKRIER5//PGM59999x1dunShYsWK\nDBo0iGeffZZSpUoFMELLKjxsjcQKCnXq/H979x9dRXnncfz9IQmYBIgJRCo/FrI1Kr9b3GIrChZE\nCymIiliVVbbbRTygtGcFW+vZVU5PdReLYRdZf1CVo0KXhtpFagWWqkhLNSASKRTCImAQBUkqDeFH\nEr77x0wuNyGBwDWZIN/XOffcuc88M/c7k2S+M89knqcPK1eupG/fvtx///1069aNYcOGsXv3bgB2\n7drFpk2b/IrFuTPgicSdEyQxdOhQli9fztq1a7n55ps5ePAgGRkZAMydO5fevXuTlpZG9+7dueaa\na5g0aRJHjx4FYP/+/Z5knGtAk45HIulbwGwgCZhnZo/WU2cc8BBgwAYzuy0s/3cgjyDZrQCmAqnA\nL4EvA9XAK2b2w1PF4eORuFPZunUra9eupbi4mG3btlFcXMynn35KcXExkrj99ttZuHAh3bp1Izc3\nl4suuoi+ffsyefJkAI4dOxZ7aNK5L4rIB7aSlARsBYYDJUAhcKuZbYqrkwssAoaaWZmkC8xsr6Qr\ngJnA4LDqauBHwDvA5Wb2uqTWwErgp2b225PF4onEJWr58uWsWbMmlmSKi4vp0qULRUVFAHzzm99k\n+/bt5ObmxhLNgAEDYl3jO3c2amwiacqb7QOBbWa2PQzoF8D1wKa4Ov8EPGFmZQBmtjcsN+A8oDUg\nIAX4xMwqgNfDukclvQt0bcJtcA4Iuma59tpra5VVVFTEpseMGUNhYSHbtm1j0aJFlJaWMnLkyFgi\nGTx4MK1bt44lmdzcXPr160ePHj2aczOcaxJNmUi6AB/GfS4BLq9T52IASb8naP56yMxeM7M1kl4H\n9hAkkjlmtjl+QUnnA6MIms5OIGkiMBGCh9ac+7ylpaXFpqdOnVprXmlpaayXYjMjJyeHLVu2xJIM\nwMSJE3nqqacwM5YuXcqwYcNqrdO5s0XU//6bDOQCVxNcWayS1BfoCPTk+NXGCklXmdlbAJKSgYXA\nf9Rc8dRlZk8DT0PQtNWUG+FcXVlZWWRlZQHBjf758+fH5pWWlrJt2zbatWsHwPr16xk9ejRpaWnk\n5eUxduxYRo4cSdu2bSOJ3bnT1ZR3B3cD3eI+dw3L4pUAS8ys0sw+ILinkgvcAPzRzMrNrBz4LfCN\nuOWeBorNLL/JoneuiWRlZTFw4EB69uwJQL9+/Vi5ciV33nknq1at4pZbbiE7O5s1a9ZEHKlzjdOU\niaQQyJWUE94Y/w6wpE6dXxNcjSCpI0FT13ZgFzBEUrKkFGAIsDms9xMgA/h+E8buXLNJTk5m6NCh\nzJ07l927d/Pmm29y11130b9/fwBmzpzJqFGjmD9/PmVlZRFH69yJmiyRmFkVMAVYRpAEFpnZnyTN\nkDQ6rLYM2C9pE8FN9Glmth8oAP4PeB/YQPBvwa9I6gr8GOgFvCvpPUnfa6ptcK65JSUlMXjwYPLz\n82P3S1JSUtiwYQMTJkzgggsuYMSIEbzwwgsRR+rccU36HElL4f/+6852ZkZhYSEFBQUUFBTUGsd+\n8eLFXHnllXTq1CniKN0XTeTPkbQknkjcF4mZceDAATIyMvjoo4/o0qULrVq1YvDgwYwdO5YbbriB\nzp07Rx2m+wJobCLxR3GdO8tIinXtcuGFF1JUVMSDDz7I3r17mTJlCl27dqWgoADAx2NxZ8TM2Ldv\nX6PrR/3vv865BEiib9++9O3bl4cffphNmzbFmroAnn32WZ555hnGjh3LTTfdRE5OTsQRu+ayd+9e\n9u/fT2lpKaWlpZSVlZGdnc2IESOA4DmmkpKSWvPHjBnDM888A8C8efMa/V2eSJz7AunVqxe9evWK\nfW7fvj1VVVVMmzaNadOmcdlllzF27FimT5/ufYO1cDVNmGVlZbGDvSSGDRsGQH5+Phs3bowlgdLS\nUi655BIWLVoEBL0pbNmypdY6r7vuulgi2bJlCxUVFWRlZZGTk0NmZiaDBg0CghOUG2+8kQceeKBR\nsfo9EufOAR988AGLFy+moKCA6upqCgsLAVi0aBG9evWiW7dupKamkpKSgqSIoz37mBlHjx6lvLyc\nDh06ALB9+3Z27NhBeXk55eXlHDx4kKNHj8Y6+pw3bx5vvfVWbH55eTnp6eksX74cgLy8PF599dVa\n35Obm8vWrVsBGDFiBEVFRWRmZsYegO3Xrx8zZswAgn/CqKysJCsrK1anY8eOsWbRxvCb7XE8kTh3\n3KFDh0hNTeXw4cN06NChVp9hSUlJ3HvvvcyaNYvKykr69+9PWloaaWlppKamkpaWxrhx47j11lup\nqKhgxowZsfKa94EDB9KnTx8OHz7M2rVra81LTU0lIyODNm3aJLwdx44d48iRI1RXV1NVVRV7ZWZm\n0qZNGw4cOMCePXuoqqqqVad3796kp6dTUlLC5s2bay1bWVlJXl4e6enprF69mtdee61WIigvL2fB\nggW0a9eOxx57jNmzZ8fm1wzpfOTIEVq3bs2UKVN44oknasWckpISG5pg6tSpLFmyhPT0dNq2bUvb\ntm350pe+xIsvvggESb6kpKRWosjOzubSSy9NeN81VkvotNE51wKlpqYCcN5551FcXMyyZcsoKyvj\n0KFDVFRUcPnlQZd41dXV9O7dm4qKCg4dOsRnn33Gxx9/HLsJe+DAAfLz8zly5Eit9c+cOZM+ffrw\n4YcfctVVV53w/XPnzuXuu++mqKiIIUOGxBJVcnIyVVVVzJo1i1GjRrF69WpuvPHGWgf6qqoqXn75\nZfLy8li6dCnXX3/9Cet/4403GDJkCK+88grjx48/Yf66desYMGAAv/nNb5g0adIJ87ds2cLFF1/M\n22+/zSOPPBI7yNe8Dh8+TLt27cjJyWH48OEnzK8xZcoUxo0bV2teeno6ZoYkZs+ezezZ9XYVCMC4\nceManNfS+BWJcy4h1dXVHD58mIqKCioqKmjfvj2ZmZkcPHiQP/zhD7FEVPM+ZMgQ+vTpw86dO/nZ\nz34Wm1dZWUlKSgqTJ0/miiuuYOvWreTn55OcnBx7JSUlcccdd9CzZ0+2bdtGQUFBrfnJycmMHj2a\nzp07s2PHDtasWUNSUlKt+YMGDSIjI4M9e/awffv2WutOTk4mNzeXNm3aUF1dTatWrc7ppj5v2orj\nicQ5506fP0finHOuWXgicc45lxBPJM455xLiicQ551xCPJE455xLiCcS55xzCfFE4pxzLiGeSJxz\nziXknHggUdI+YGfEYXQEPo04hpbC98Vxvi+O831xXEvZF93NLPtUlc6JRNISSFrbmCdEzwW+L47z\nfXGc74vjzrZ94U1bzjnnEuKJxDnnXEI8kTSfp6MOoAXxfXGc74vjfF8cd1btC79H4pxzLiF+ReKc\ncy4hnkicc84lxBNJE5PUTdLrkjZJ+pOkqVHHFCVJSZLWS1oadSxRk3S+pAJJf5a0WdI3oo4pKpJ+\nEP59bJS0UNJ5UcfUXCQ9K2mvpI1xZVmSVkgqDt8zo4zxVDyRNL0q4J/NrBfwdWCypF4RxxSlqcDm\nqINoIWYDr5nZpUB/ztH9IqkLcC/wd2bWB0gCvhNtVM3qeeBbdcp+CKw0s1xgZfi5xfJE0sTMbI+Z\nvRtO/5XgYNEl2qiiIakrkAfMizqWqEnKAAYDPwcws6Nm9pdoo4pUMpAqKRlIAz6KOJ5mY2argNI6\nxdcD88Pp+cCYZg3qNHkiaUaSegBfBd6ONpLI5APTgWNRB9IC5AD7gOfCpr55ktKjDioKZrYbeAzY\nBewBPjOz5dFGFblOZrYnnP4Y6BRlMKfiiaSZSGoLLAa+b2YHoo6nuUn6NrDXzNZFHUsLkQwMAP7L\nzL4KHKSFN180lbD9/3qC5NoZSJc0PtqoWg4LntFo0c9peCJpBpJSCJLIS2b2q6jjicggYLSkHcAv\ngKGSXow2pEiVACVmVnN1WkCQWM5F1wAfmNk+M6sEfgVcEXFMUftE0oUA4fveiOM5KU8kTUySCNrB\nN5vZrKjjiYqZ/cjMuppZD4Ibqb8zs3P2rNPMPgY+lHRJWDQM2BRhSFHaBXxdUlr49zKMc/QfD+Is\nAe4Mp+8E/ifCWE7JE0nTGwT8PcEZ+Hvha2TUQbkW4R7gJUlFwFeAn0YcTyTCq7IC4F3gfYLj0lnV\nRUgiJC0E1gCXSCqR9I/Ao8BwScUEV2yPRhnjqXgXKc455xLiVyTOOecS4onEOedcQjyROOecS4gn\nEueccwnxROKccy4hnkjcaZHUSdICSdslrZO0RtINTfRdEyTNCacnSbojnH5e0tjTWM+3w25INoS9\nMN/V2O89g5gfOINl6v2+cF8vjYv71Uasa4ekjqcbQ5119JB02xksNzPswXdmnfJ6t0NSZ0kFicQa\nrqc80XW4xCRHHYA7e4QPi/0amG9mt4Vl3YHR9dRNNrOqz+u7zezJM1ku7FXgaWCgmZVIagP0+Lzi\nqscDfH7Pg8wAVpjZbABJ/T6n9Z5KD+A2YMFpLjcRyDKz6jrl9W6HmX0ENPqEwLVcfkXiTsdQ4Gj8\nQd3MdprZf0LszHqJpN8RdH2NpGmSCiUVSXq4ZjlJ4yW9Ez6g+ZSkpLD8HyRtlfQOwcOcNfUfknRf\nfDCShkr6ddzn4ZJerhNzO4ITpv1hvEfMbEtYP1vS4jC+QkmD6izbYB1JbSU9J+n9cNtukvQoQQ+2\n70l66Uy2s44LCbpSqdnXReGyVytuPBdJcyRNiFtuehjXO5IuCuvcrGCsjw2SVoVlSeFVRM3Pp+ZK\n7VHgqjDmH9TZHwqX2Rh+xy1h+RKgLbCupqwR29FD4RgcCjqtrHlgd5+kfw3L6/39qY+kjgqukPNO\nVs81ATPzl78a9SIYM+Lxk8yfQHDAyAo/X0twNSCCk5alBF2n9wReAVLCenOBOwgOOLuAbKA18Htg\nTljnIeC+cPp5gjNZAX8GssPyBcCoeuKaR9BX0ULgdqBVXP0rw+m/IejGpmY75pyizr8B+XHfkRm+\nl8eVnfZ21on7OuAvwOvAj4HOYfnVwNK4enOACeH0DuDH4fQdNfUInhjvEk6fH75PBB4Mp9sAawk6\nTqy1/jox3QSsIBgzpFO4HRfW3fZGbkcPYGOdut0JukfpTgO/P/WsvzyM5W1geNR/J+fiy5u23BmT\n9ARwJcFVytfC4hVmVjO2wrXha334uS2QC/QDLgMKg9YyUgkO9JcDb5jZvnD9/w1c3ND3m5lJegEY\nL+k54BsEB8+69b4nqS9BVxP3AcMJksU1QK8wBoD2CnppjtdQnWuIG3zJzMrqCXFYIttpZssk/S3B\noEcjgPWS+jS0P+IsjHt/PJz+PfC8pEUEnSJC8LPpp+P3mzIIfj5HT7LuK4GFFjRffSLpTeBrBH1D\n1aux26FgVMRfAveY2U5J91D/78+qOoumEFwBTzazN08Su2sinkjc6fgTwRkpAGY2WcGN3bVxdQ7G\nTQt4xMyeil9JeICYb2Y/qlN+JoP3PEdw1n8Y+KU1cF/GzN4H3g8TzwcEiaQV8HUzO1wnjviPjanT\nEJHgdoZJeQGwIGzOGgx8Qu1m6brD0lrdaTObJOlygoHF1km6LIzvHjNbVie+qxsbX2M1sB11hxR4\nEviVmf1vTSjU8/tTj6pwXdcBnkgi4PdI3On4HXCepLvjytJOUn8Z8N2as3xJXSRdQHD2ODacrhmf\nujtB08QQSR0U3CS/+VQBWXDD9iPgQYKkUkt4L+PquKKvADvD6eUEHSfW1P1KPV/RUJ0VwOS48pox\ntSvD2El0O8N7QGnhdDvgywRNSTsJrpLaSDqf4Mon3i1x72vC5b9sZm+b2b8QDKjVjeDnc3dNvJIu\nVjC41l8J7i3V5y3glvD+SjZBQningbqn2o74OpOBdmYW3zlhQ78/dRnwXeBSSfefLBbXNPyKxDVa\n2JQ0Bnhc0nSCA9JBoN4/XjNbLqknsCY8gy8HxpvZJkkPAssltQIqCZol/ijpIYKD31+A9xoZ2ksE\n90nq63pcBDefnwIOhfFOCOfdCzyhoPfdZIImk0l1lm+ozk/C8o1ANfAwQZPR00CRpHfN7PYEt/My\nYI6kKoKTvnlmVggQNlFtJLi6Wl9nucww3iPArWHZTEm54f5YCWwAigjuU7yr4Ae0j2BI1yKgWtIG\n4Hkzezxu3S8TNCFuIDiAT7egS/yTqXc7FIwYWuM+giRcsy+eNLMn6/v9oZ6xOcysWtKtwBJJfzWz\nuaeIyX2OvPdfd9ZT8AzGejP7edSxOHcu8kTizmqS1hFcZQw3syNRx+PcucgTiXPOuYT4zXbnnHMJ\n8UTinHMuIZ5InHPOJcQTiXPOuYR4InHOOZeQ/weO9UHr88KhOAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Now plot the incremental results\n",
    "ks = []; mus = []; serrs = [];\n",
    "for i in range(len(r.keys())):\n",
    "    ks.append(i+1)\n",
    "    mus.append(r[i+1][1])\n",
    "    serrs.append(r[i+1][2])\n",
    "\n",
    "    \n",
    "best_1serr = min(np.array(mus) + np.array(serrs))\n",
    "plt.clf()\n",
    "plt.plot(ks, mus, 'b.-', label = 'LL of Set k')\n",
    "plt.plot(ks, np.array(mus) + np.array(serrs), 'k+-')\n",
    "plt.plot(ks, np.array(mus) - np.array(serrs), 'k--')\n",
    "plt.plot(ks, np.ones(len(ks))*best_1serr, 'r', label ='1 std err rule')\n",
    "\n",
    "plt.xlim([1,11])\n",
    "\n",
    "plt.title('Stepwise Forward Feature Selection')\n",
    "plt.xlabel('Greedily Selected Subset of Size k')\n",
    "plt.ylabel('X Validated LogLoss')\n",
    "    \n",
    "plt.legend(loc=1, ncol=2)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>The above plot will support any of the stopping criteria defined above. Assuming we haven't predefined $k$, we might choose $k=7$, as that is where the error really seems to plateau. Additionally, we could be more conservative. The red line shows the min value of (mean+1 std error). Any value of $k$ that is less than this value is statistically the same as the $k$ with the lowest mean error. If we were using that rule, we'd potentially stop at $k=3$.<br><br>\n",
    "</p>\n",
    "\n",
    "## SVD Based Dimensionality Reduction\n",
    "\n",
    "<p>Another way to bring the learning down to $k$ features is to find a projection matrix that produces a rank-$k$ approximation of our training data $X$. The best way to get a low-rank approximation, from a signal preservation perspective, is by the use of the Singular Value Decomposition. A review of the SVD can be found here:<br><br>\n",
    "http://nbviewer.ipython.org/github/briandalessandro/DataScienceCourse/blob/master/ipython/Lecture3_PhotoSVD.ipynb\n",
    "<br><br>\n",
    "We'll put it to work here by doing the following:\n",
    "<ul>\n",
    "    <li>Decomposing our training data</li>\n",
    "    <li>For each rank-k approximation of X, get cross-validated error</li>\n",
    "    <li>Compare this error to the Stepwise Forward Feature Selection above</li></ul><br>\n",
    "The scale of the features influences the spectrum of the singular values, so we'll normalize the feature so they all have the same scale.\n",
    "</p>    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import math\n",
    "from sklearn import linear_model\n",
    "from sklearn.preprocessing import scale\n",
    "\n",
    "\n",
    "f = '../data/Cell2Cell_data.csv'\n",
    "\n",
    "X_train = train.drop(lab, 1)\n",
    "X_test = test.drop(lab, 1)\n",
    "Y_train = train[lab]\n",
    "Y_test = test[lab]\n",
    "\n",
    "X_train_norm = pd.DataFrame(scale(X_train, axis=0, with_mean=True, with_std=True, copy=True), columns = X_train.columns.values)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Now let's decompose the training data.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "U, sig, Vt = np.linalg.svd(X_train_norm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Out of curiosity, let's plot the spectrum to get a sense of how independent the features are."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl8VdW5//HPQ0JAZpAAARICyhRA\nBiMIXKtVqaAo3tpacKhIW6oVx062dai0v2pb61RpvVQRrQMO9VpULFacKg4lzBAICQgkBEkCkjBn\nen5/JPWmKZIDnJOdc873/Xrx4uy9F9nPweTrYp211zJ3R0REYkuzoAsQEZHwU7iLiMQghbuISAxS\nuIuIxCCFu4hIDFK4i4jEIIW7iEgMUriLiMQghbuISAxKDOrGnTt39vT09KBuLyISlZYuXVri7skN\ntQss3NPT08nKygrq9iIiUcnMtoTSTsMyIiIxSOEuIhKDFO4iIjFI4S4iEoMU7iIiMajBcDezOWZW\nZGZrvuC6mdlDZpZnZqvMbET4yxQRkaMRSs99LjD+CNcnAH1rf00H/nj8ZYmIyPFocJ67u79nZulH\naDIJeNJr9uv7yMw6mFmKu28PU40iIlHN3dleepDswjLWFpZxzsAuDO7RPqL3DMdDTD2A/DrHBbXn\n/iPczWw6Nb170tLSwnBrEZGmpara+aRkL2trg7wm0Ev5bH8FAGbQqU1SVIR7yNx9NjAbIDMzUztz\ni0hUO1hRRc6ne2qDvJS1hWWs/7SMgxXVACQlNKN/t7acN6gbGd3bMah7OwZ0a0frFpGP3nDcYRuQ\nWue4Z+05EZGYsXt/OdmFZWRvL/s8zDcW76Oquqaf2rZlIhkp7bhsZC8GdW/HoB7tOCm5Dc0TgpmU\nGI5wnw/MMLN5wCigVOPtIhKt/jU+Xrc3nl1YxrbdBz5v061dSwZ1b8f4z3vk7enZ8QTMLMDK/12D\n4W5mzwJnAZ3NrAC4E2gO4O6PAAuA84E8YD9wdaSKFREJp6pqZ1Nxzfh4TY+8lOzCsn8bH+/TuTUj\nenXkytE1PfKMlHac2KZFwJU3LJTZMlMauO7AdWGrSEQkQtydddv3sGD1dt7PK/n38fHEZgzo1pbx\ng7uR0b09GSntGJjSllZJgS2ee1yis2oRkRC5Ozk79vDaqu28tmo7m0r20cwgs1cnLh9VOz7evT0n\nJbcmMaDx8UhQuItITNqwYw+vrtrOa6sK2VhcE+ijTzqRb5/Rh/MGdY2KoZXjoXAXkZiR+69AX72d\nvKK9NDMY1ftErh7bm/GDu9E5xgO9LoW7iES1vKK9NUMuqwvZsGMvZjCqdyeuGj2I8wZ3o0vblkGX\nGAiFu4hEnY3Fe1lQ20Nf/+kezOC09E7MnDSI8YO60aVdfAZ6XQp3EYkKn5Ts47VVhby6qibQAU5L\n78jPL8xgwpAUuirQ/43CXUSarM0l+3htdc0sl+ztZQCc2qsjd0zM4PwhKXRrr0D/Igp3EWlStuz8\nv0BfW1gT6CPSOnD7xAwmDO5G9w4nBFxhdFC4i0jg8nft/zzQV28rBWBYagduu2AgE4ak0EOBftQU\n7iISiNL9FbywNJ9XVhaysqAm0IemduBn5w9kwpBu9OzYKuAKo5vCXUQaVe6OPTz+wWb+d9k2DlRU\nMaRHe34yYQDnD0khtZMCPVwU7iIScVXVztvri5j7wWbezyshKbEZFw/rztQxvcno3i7o8mKSwl1E\nIqbsYAUvZBXwxAeb2bprP93ateSH5/Vnysg0OrVOCrq8mKZwF5Gw21i8lyc+2MyLSwvYX15FZq+O\n/Hj8AL4yqGtgm1fEG4W7iIRFdbXzbm4xcxdv5t0NxSQlNOPCod2ZOiadIT0ju1+o/CeFu4gcl72H\nKvnL0pqhl00l++jStgW3jOvHlJFpJLeNn4W6mpqQwt3MxgMPAgnAo+5+T73rvYA5QDKwC7jC3QvC\nXKuINCGbS/bxxIebeSGrgL2HKhmW2oEHJw9jwuAUkhI19BK0ULbZSwBmAeOAAmCJmc139+w6ze4F\nnnT3J8zsbOBu4MpIFCwiwXF33s8rYe7izbyVU0RiM+OCISlcNSad4Wkdgy5P6gil5z4SyHP3TQC1\nG2FPAuqGewZwS+3rt4GXw1mkiARrf3klf1m2jSc+2Exe0V46t0ni+rP7csWoNK3A2ESFEu49gPw6\nxwXAqHptVgJfpWbo5r+BtmZ2orvvDEuVIhKI/F37efLDzTy3JJ+yg5UM6dGe+y4dygWnpNAiMSHo\n8uQIwvWB6g+Ah81sKvAesA2oqt/IzKYD0wHS0tLCdGsRCSd358NNO3l88WbeXLeDZmZMGNyNq8em\nMyKtI2YWdIkSglDCfRuQWue4Z+25z7l7ITU9d8ysDXCJu++u/4XcfTYwGyAzM9OPsWYRiYAD5VW8\nvGIbcxdvJmfHHjq1TuK6s07m8tPTSGmvhbuiTSjhvgToa2a9qQn1ycBldRuYWWdgl7tXAz+hZuaM\niESBHWUHeXzxZuYt2cru/RUMTGnHb752ChcN7U7L5hp6iVYNhru7V5rZDGAhNVMh57j7WjObCWS5\n+3zgLOBuM3NqhmWui2DNIhIGhbsP8Mi7G5m3JJ/KqmrOG9SNqWPSGdm7k4ZeYoC5BzM6kpmZ6VlZ\nWYHcWySe5e/azx/f3cgLWfm4w9cze3LtmSeTdqJWZIwGZrbU3TMbaqcnVEXixJad+5j1dh4vLdtG\nMzO+cVoq15x5ktZNj1EKd5EYt7F4L7PezuOvKwpJbGZccXovvntmH31IGuMU7iIxKnfHHn7/Vh6v\nriokKbEZV49JZ/qX+uihozihcBeJMeu2l/HwW3ksWLOdE5on8J0v9eE7Z/Shcxst4hVPFO4iMWLN\ntlIeWpTLG9k7aNMikevOOplp/9Vbm2LEKYW7SJRbkb+b3y/KZdH6Itq2TOTGc/oybWxv2rdqHnRp\nEiCFu0iUWrplFw8tyuPdDcV0aNWcH3ylH98ck067lgp1UbiLRJ2PN+3kobdyWZy3k06tk/jx+AFc\nOboXbVrox1n+j74bRKKAu/Phxp08uCiXjz/ZRec2LbjtgoFcNiqNVkn6MZb/pO8KkSbM3Xkvt4SH\nFuWydMtndG3XgjsvzGDKyDSt+yJHpHAXaYLcnbfWF/HQolxWFpTSvX1LfnHxYL5+ak+FuoRE4S7S\nhFRXO39ft4OHFuWytrCMnh1P4O6vDuGSET21L6kcFYW7SBNQXe0sXPspDy7KZf2ne0g/sRW//dop\nXDy8B80TFOpy9BTuIgFyd97ZUMy9C3NYW1hGn+TW3P+NoVx4SncSFepyHBTuIgH5aNNO7l2YQ9aW\nz0jtdAK/+/pQLh7eg4RmWktdjp/CXaSRrczfzb1v5PCP3BK6tmvBLy8ezKWZqRpTl7BSuIs0kpxP\n9/C7N3J4I3sHHVs152fnD+TK0b00+0UiIqRwN7PxwIPUbLP3qLvfU+96GvAE0KG2za3uviDMtYpE\npc0l+3jgzQ38dWUhbZISuWVcP6b9V289USoR1eB3l5klALOAcUABsMTM5rt7dp1mtwHPu/sfzSwD\nWACkR6BekaixvfQADy3K4/msfJonGN/90kl890t96KhVGqURhNJ1GAnkufsmADObB0wC6oa7A+1q\nX7cHCsNZpEg0Kdl7iD+8vZGnPt6Cu3PFqDSu+/LJ2iRDGlUo4d4DyK9zXACMqtfm58AbZnY90Bo4\nNyzViUSR0gMV/Om9TcxZ/AkHK6q4ZERPbjinL6mdtEepNL5wDfpNAea6++/MbDTwZzMb7O7VdRuZ\n2XRgOkBaWlqYbi0SrP3llTy+eDP/8+5Gyg5WMvGUFG4e14+TktsEXZrEsVDCfRuQWue4Z+25ur4F\njAdw9w/NrCXQGSiq28jdZwOzATIzM/0YaxZpEg5WVPHMx1v5wzt5lOwt55wBXbjlK/0Y1L190KWJ\nhBTuS4C+ZtabmlCfDFxWr81W4BxgrpkNBFoCxeEsVKSpqKyq5sWlBTy0KJfC0oOM7nMi/3Nlf07t\n1THo0kQ+12C4u3ulmc0AFlIzzXGOu681s5lAlrvPB74P/MnMbqbmw9Wp7q6eucSU6mrnlVWFPPBm\nLp+U7GNYagd++/WhjD25c9ClifyHkMbca+esL6h37o46r7OBseEtTaRpcHfeXFfE797IYf2nexjQ\nrS2PfjOTcwZ2wUxLBUjTpKcoRI5gcV4Jv1mYw8r83fTu3JqHpgxn4pAUmmn9F2niFO4ih7F0y2fc\nuzCHDzftpHv7lvz6kpo11bVSo0QLhbtIHbk79nDP6+tZtL6Izm2S+PmFGUwZlUaLRK3/ItFF4S4C\n7N5fzgNv5vLnj7bQOimBH43vz9Qx6dp8WqKWvnMlrlVWVfPsP7fyu79voOxABZeNSuOWcf3ppPVf\nJMop3CVuLc4rYeYr2eTs2MPoPidy50UZDOjWruE/KBIFFO4Sd7bs3Mf/e20db2TvILXTCTxyxamc\nN6irpjVKTFG4S9zYe6iSh9/KY877n5CYYPxofH+mje2tzTIkJincJeZVVzt/WVbAbxbmULznEJeM\n6MmPxvenq5bglRimcJeYtnTLLu56JZtVBaUMT+vAn76ZybDUDkGXJRJxCneJSdtLD3DP6+v564pC\nurZrwQPfGMZFQ7vryVKJGwp3iSkHK6qY/d4m/vjORqrduf7sk7nmzJNorf1KJc7oO15igrvz2urt\n3L1gPdt2H+CCISncOmGAdkGSuKVwl6i3ZlspM1/J5p+bdzEwpR2/u3Qop/c5MeiyRAKlcJeoVbL3\nEPcuzOG5rHw6tkri7q8O4dLMVBI0ri6icJfoU15ZzRMfbOahRbkcqKjiW2N7c/05fWl/QvOgSxNp\nMhTuEjXcnbfWF/HL19bxSck+zh7QhZ9dMFAbUYscRkjhbmbjgQep2WbvUXe/p971+4Ev1x62Arq4\nuyYTS9jkFe1h5qvreG9DMX2SW/P41afx5f5dgi5LpMlqMNzNLAGYBYwDCoAlZja/dms9ANz95jrt\nrweGR6BWiUOl+yu4/80N/PmjLbRKSuD2iRl8c3QvmmvTDJEjCqXnPhLIc/dNAGY2D5gEZH9B+ynA\nneEpT+LVv5bive/vGyg9UMGUkWncMq4fJ7ZpEXRpIlEhlHDvAeTXOS4ARh2uoZn1AnoDbx1/aRKv\nNhbv5ebnVrCqoJTT+3TijomDyOiupXhFjka4P1CdDLzo7lWHu2hm04HpAGlpaWG+tUQ7d+eZf27l\nl6+uo0XzZjx82XAuGJKipXhFjkEo4b4NSK1z3LP23OFMBq77oi/k7rOB2QCZmZkeYo0SB3buPcSP\n/7KaN9ft4Iy+nbn360O1aqPIcQgl3JcAfc2sNzWhPhm4rH4jMxsAdAQ+DGuFEvPeySnihy+uonR/\nBbdPzODqMela4EvkODUY7u5eaWYzgIXUTIWc4+5rzWwmkOXu82ubTgbmubt65BKSgxVV3PP6euZ+\nsJn+Xdvy5LSRDEzR2LpIOIQ05u7uC4AF9c7dUe/45+ErS2Lduu1l3DhvORt27OXqsen8ePwA7Ygk\nEkZ6QlUaVXW1M2fxJ/zmbzm0b9WcJ6aN5Mx+yUGXJRJzFO7SaHaUHeT7z6/k/bwSxmV05Z6vDtG8\ndZEIUbhLo/jbmu3c+tJqDlVU86v/HsKUkama4igSQQp3iah9hyq565W1PJ9VwJAe7Xlw8jD6aKEv\nkYhTuEvErMjfzU3zlrNl136+d9ZJ3HRuP5IStSaMSGNQuEvYVVZV84d3NvLgoly6tWvJvO+czijt\njCTSqBTuElb5u/Zz83MryNryGZOGdWfmpMHaREMkAAp3CQt353+Xb+OOv67FgAe+MYyLh/cIuiyR\nuKVwl+NWeqCC215ewysrCzktvSP3XTqM1E6tgi5LJK4p3OW4fLRpJ7c8t4KiPYf44Xn9uebMk7RB\ntUgToHCXY1JeWc39b27gkXc3kn5ia/5y7RiGpmpnRZGmQuEuRy2vaC83PbecNdvKmDIyldsuyKB1\nC30riTQl+omUkLk7T3+8lV++ls0JzRN45IpTGT+4W9BlichhKNwlJDWbaazizXVF2kxDJAoo3KVB\nb+cU8cMXVlF2UJtpiEQLhbt8ofqbaTz17ZEM6KbNNESigcJdDit/136ue2YZqwpKtZmGSBQKaRUn\nMxtvZjlmlmdmt35Bm0vNLNvM1prZM+EtUxrT2zlFXPjw+3xSso/ZV57KnRcOUrCLRJkGe+5mlgDM\nAsYBBcASM5vv7tl12vQFfgKMdffPzKxLpAqWyKmqdh5clMvv38qlf9e2PHLFqaR3bh10WSJyDEIZ\nlhkJ5Ln7JgAzmwdMArLrtPkOMMvdPwNw96JwFyqRtWtfOTfOW84/cku4ZERPfnnxYE5IUm9dJFqF\nEu49gPw6xwXAqHpt+gGY2WIgAfi5u/+t/hcys+nAdIC0tLRjqVciYPnWz7ju6WWU7Cvn7q8OYfJp\n2iVJJNqF6wPVRKAvcBbQE3jPzIa4++66jdx9NjAbIDMz08N0bzlG7s5TH21h5qvZdG3Xkr9cM4Yh\nPdsHXZaIhEEo4b4NSK1z3LP2XF0FwMfuXgF8YmYbqAn7JWGpUsJuf3klP31pNS+vKOTL/ZO5/xvD\n6NAqKeiyRCRMQpktswToa2a9zSwJmAzMr9fmZWp67ZhZZ2qGaTaFsU4Jo43Fe7l41mL+urKQH3yl\nH49ddZqCXSTGNNhzd/dKM5sBLKRmPH2Ou681s5lAlrvPr732FTPLBqqAH7r7zkgWLsdmwert/OjF\nVSQlNuPJaSM5o29y0CWJSASYezBD35mZmZ6VlRXIveNRRVU1v359PY++/wnDUjvwh8tH0L3DCUGX\nJSJHycyWuntmQ+30hGoc2FF2kBnPLGPJ5s+YOiadn54/kKTEkJ5fE5EopXCPcR9u3Mn1zy5j36Eq\nHpw8jEnDtK+pSDxQuMcod+eRdzfx24XrSe/cmme+czr9urYNuiwRaSQK9xhUeqCCH7ywkr9n7+CC\nISn8+mun0EY7JYnEFf3Ex5jswjKufXop2z47wO0TM5g2Nl1Pm4rEIYV7DHkhK5/bXl5Dh1bNmTf9\ndDLTOwVdkogEROEeAw5WVHHXK2t59p/5jO5zIg9NGU5y2xZBlyUiAVK4R7n8Xfu59umlrNlWxrVn\nncT3x/UjMUHTHEXincI9ir29voibnltBtTt/+mYm4zK6Bl2SiDQRCvcoVFXtPPjmBh56K4+BKe14\n5IoR9DpRm2qIyP9RuEeZnXsPcdNzK/hHbglfP7Unv7h4sLbAE5H/oHCPIstqN9XYua+cX18yhG+c\npg1PROTwFO5RwN158sMt/PK1mk01Xrp2DIN7aFMNEfliCvcmzt2Z+Wo2jy/ezNkDunD/pcNo36p5\n0GWJSBOncG/iHl+8mccXb2bqmHTumJhBs2Z62lREGqYJ0U3Ym9k7+MVr2Xwloyu3K9hF5CiEFO5m\nNt7Mcswsz8xuPcz1qWZWbGYran99O/ylxpc120q5Yd5yBndvzwOTh5GgYBeRo9DgsIyZJQCzgHHU\nbIS9xMzmu3t2vabPufuMCNQYd7aXHuBbTyyhwwnNeeyqTFolafRMRI5OKD33kUCeu29y93JgHjAp\nsmXFr32HKvnW3Cz2Harisamn0aVdy6BLEpEoFEq49wDy6xwX1J6r7xIzW2VmL5pZaliqizNV1c4N\nzy5n/adl/P6y4QxMaRd0SSISpcL1georQLq7nwL8HXjicI3MbLqZZZlZVnFxcZhuHTt+8Wo2i9YX\ncddFg/hy/y5BlyMiUSyUcN8G1O2J96w99zl33+nuh2oPHwVOPdwXcvfZ7p7p7pnJycnHUm/MeuKD\nzcz9YDPTxvbmytHpQZcjIlEulHBfAvQ1s95mlgRMBubXbWBmKXUOLwLWha/E2PfW+h3c9cpazh3Y\nlZ9dMDDockQkBjQ4DcPdK81sBrAQSADmuPtaM5sJZLn7fOAGM7sIqAR2AVMjWHNMyS4s4/pnljMw\npR0PasqjiISJuXsgN87MzPSsrKxA7t1U7Cg7yMWzFuMOL183lm7tNTNGRI7MzJa6e2ZD7TSBOiD7\nyyv51hNLKD1QwQvXjFawi0hYafmBANRMeVxBdmEZD182nEHdtcKjiISXwj0Av1qwjjfX7eCOiRmc\nPUBb44lI+CncG9mfP9rCY+9/wtQx6Uwd2zvockQkRincG9E7OUX8fP5azh7QhdsnZgRdjojEMIV7\nI1n/aRkznllOv65teWjKcE15FJGIUrg3gqKyg0x7fAmtWyQwZ2ombVpokpKIRJZSJsL2l1fy7Sez\n+Gx/zZTHlPYnBF2SiMQBhXsEVVc7Nz+3gtXbSpl9ZaY2tRaRRqNhmQi652/rWbh2B7ddkMG4DE15\nFJHGo3CPkGc+3srs9zZx5em9mDY2PehyRCTOKNwj4L0Nxdz+1zWc1T+ZOy/MwEwzY0SkcSncwyzn\n0z1c9/Qy+nZpw++nDCcxQX/FItL4lDxhVLznENPmLqFlUgKPTT2Nti2bB12SiMQphXuYHCiv4ttP\nZrFz3yEeuyqTHh005VFEgqOpkGFQXe3c8vwKVhXs5pErTuWUnh2CLklE4px67mHwm4U5vL7mU346\nYSDnDeoWdDkiIqGFu5mNN7McM8szs1uP0O4SM3Mza3CXkFgx759beeTdjVw2Ko1vn6FVHkWkaWgw\n3M0sAZgFTAAygClm9h9LGppZW+BG4ONwF9lULc4r4baX13BG387cddEgTXkUkSYjlJ77SCDP3Te5\nezkwD5h0mHa/AH4NHAxjfU1W7o49XPPUUvokt2bW5SNorimPItKEhJJIPYD8OscFtec+Z2YjgFR3\nfy2MtTVZJXsPcfXcJbRITGDO1NNopymPItLEHHd308yaAfcB3w+h7XQzyzKzrOLi4uO9dSAOVlTx\nnSezKNl7iEevyqRnx1ZBlyQi8h9CCfdtQGqd45615/6lLTAYeMfMNgOnA/MP96Gqu89290x3z0xO\nTj72qgNSXe18/4WVLN+6m/svHcawVE15FJGmKZRwXwL0NbPeZpYETAbm/+uiu5e6e2d3T3f3dOAj\n4CJ3z4pIxQG67+8beG3Vdm6dMIAJQ1KCLkdE5As1GO7uXgnMABYC64Dn3X2tmc00s4siXWBT8frq\n7Tz8dh6TT0vlu1/qE3Q5IiJHFNITqu6+AFhQ79wdX9D2rOMvq2nZVLyXH764iqGpHbhrkqY8ikjT\np/l7DThQXsX3nl5G8wTjD5ePoEViQtAliYg0SGvLHIG787OXV5OzYw9zrx6pxcBEJGqo534Ez/4z\nn5eWbeOGs/tyZr/om90jIvFL4f4FVheU8vP5azmjb2duOKdv0OWIiBwVhfth7N5fzrVPL6VzmyQe\nnDychGb6AFVEoovG3OupWZt9JTvKDvL8d0fTqXVS0CWJiBw19dzr+eO7G3lrfRG3T8xgeFrHoMsR\nETkmCvc6FueV8Ls3crhoaHeuPL1X0OWIiBwzhXutT0sPcsOzy+mT3Ia7vzpEDyqJSFTTmDtQUVXN\njGeWcaCiiueuGEHrFvprEZHophQD7nl9PVlbPuP3U4Zzcpe2QZcjInLc4n5YZsHq7Tz2/idMHZPO\nhUO7B12OiEhYxHW4byrey49eXMXwtA789PyBQZcjIhI2cRvu+8srufapZSQlNmPWZSNISozbvwoR\niUFxOebu7tz2v2vYULSHJ6eNpLsWBBORGBOX3dVn/rmVl5Zv46Zz+nFGXy0IJiKxJ+7CfVXBbu6a\nn82Z/ZK5/uyTgy5HRCQiQgp3MxtvZjlmlmdmtx7m+jVmttrMVpjZ+2aWEf5Sj9/u/eVc+9Qyktu2\n4IFvDKOZFgQTkRjVYLibWQIwC5gAZABTDhPez7j7EHcfBvwGuC/slR6n6mrnpudWULTnILMuH0FH\nLQgmIjEslJ77SCDP3Te5ezkwD5hUt4G7l9U5bA14+EoMj1lv5/FOTjF3TMxgWGqHoMsREYmoUGbL\n9ADy6xwXAKPqNzKz64BbgCTg7LBUFybv55Zw35sbmDSsO1doQTARiQNh+0DV3We5+0nAj4HbDtfG\nzKabWZaZZRUXF4fr1ke0vfQAN8xbzslaEExE4kgo4b4NSK1z3LP23BeZB1x8uAvuPtvdM909Mzk5\n8lMQyyurue7pZRyqqOKPV5xKq6S4nNYvInEolHBfAvQ1s95mlgRMBubXbWBmdTcZvQDIDV+Jx+7u\n19exbOtufv21Uzi5S5ugyxERaTQNdmXdvdLMZgALgQRgjruvNbOZQJa7zwdmmNm5QAXwGXBVJIsO\nxaurCnl88Wamjkln4ilaEExE4ktI4xTuvgBYUO/cHXVe3xjmuo5LXtFefvziKkZoQTARiVMx94Tq\n/vJKvvf0Ulo0T2DW5VoQTETiU0x9wuju/PSl1eQW7eXJaSNJaa8FwUQkPsVUt/apj7fy8opCbj5X\nC4KJSHyLmXBfmb+bX7ySzVn9k5nxZS0IJiLxLSbC/bN95Xzv6ZoFwe6/VAuCiYhE/Zh7dbVz8/Mr\nKN5ziBeuGa0FwUREiIGe+8O1C4LdfmEGQ7UgmIgIEOXh/o/cYu5/cwMXD+vOFaPSgi5HRKTJiNpw\nL9x9gBvnraBvlzb8SguCiYj8m6gM9/LKaq57ZhnlldVaEExE5DCiMhV/tWAdy7fu5g+Xj+CkZC0I\nJiJSX9T13F9ZWcjcDzYzbWxvzh+SEnQ5IiJNUtSFe6fWSYzL6MpPzh8QdCkiIk1W1A3LjD25M2NP\n7hx0GSIiTVrU9dxFRKRhCncRkRikcBcRiUEhhbuZjTezHDPLM7NbD3P9FjPLNrNVZrbIzHqFv1QR\nEQlVg+FuZgnALGACkAFMMbOMes2WA5nufgrwIvCbcBcqIiKhC6XnPhLIc/dN7l4OzAMm1W3g7m+7\n+/7aw4+AnuEtU0REjkYo4d4DyK9zXFB77ot8C3j9cBfMbLqZZZlZVnFxcehViojIUQnrB6pmdgWQ\nCfz2cNfdfba7Z7p7ZnKytsETEYmUUB5i2gak1jnuWXvu35jZucDPgDPd/VBDX3Tp0qUlZrYl1ELr\n6QyUHOOfjVZ6z/FB7zk+HM97DmnCirn7kRuYJQIbgHOoCfUlwGXuvrZOm+HUfJA63t1zj7HgkJlZ\nlrtnRvo+TYnec3zQe44PjfFGKavXAAADJ0lEQVSeGxyWcfdKYAawEFgHPO/ua81sppldVNvst0Ab\n4AUzW2Fm8yNWsYiINCiktWXcfQGwoN65O+q8PjfMdYmIyHGI1idUZwddQAD0nuOD3nN8iPh7bnDM\nXUREok+09txFROQIoi7cG1rnJtaYWaqZvV27ds9aM7sx6Joag5klmNlyM3s16Foag5l1MLMXzWy9\nma0zs9FB1xRpZnZz7ff0GjN71sxaBl1TuJnZHDMrMrM1dc51MrO/m1lu7e8dI3HvqAr3ENe5iTWV\nwPfdPQM4HbguDt4zwI3UzM6KFw8Cf3P3AcBQYvy9m1kP4AZq1qQaDCQAk4OtKiLmAuPrnbsVWOTu\nfYFFtcdhF1XhTgjr3MQad9/u7stqX++h5of+SMs/RD0z6wlcADwadC2NwczaA18CHgNw93J33x1s\nVY0iETih9lmaVkBhwPWEnbu/B+yqd3oS8ETt6yeAiyNx72gL96Nd5yammFk6MBz4ONhKIu4B4EdA\nddCFNJLeQDHweO1Q1KNm1jrooiLJ3bcB9wJbge1Aqbu/EWxVjaaru2+vff0p0DUSN4m2cI9bZtYG\n+Atwk7uXBV1PpJjZRKDI3ZcGXUsjSgRGAH909+HAPiL0T/WmonaceRI1/2PrDrSuXZsqrnjNdMWI\nTFmMtnAPaZ2bWGNmzakJ9qfd/aWg64mwscBFZraZmmG3s83sqWBLirgCoMDd//UvshepCftYdi7w\nibsXu3sF8BIwJuCaGssOM0sBqP29KBI3ibZwXwL0NbPeZpZEzQcwMb3UgZkZNWOx69z9vqDriTR3\n/4m793T3dGr++77l7jHdo3P3T4F8M+tfe+ocIDvAkhrDVuB0M2tV+z1+DjH+IXId84Gral9fBfw1\nEjcJafmBpsLdK83sX+vcJABz6i5gFqPGAlcCq81sRe25n9YuCSGx43rg6dpOyybg6oDriSh3/9jM\nXgSWUTMjbDkx+KSqmT0LnAV0NrMC4E7gHuB5M/sWsAW4NCL31hOqIiKxJ9qGZUREJAQKdxGRGKRw\nFxGJQQp3EZEYpHAXEYlBCncRkRikcBcRiUEKdxGRGPT/AZ7NdCkoFXvsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(np.cumsum(sig**2)/np.sum(sig**2))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We can see above that the first 5 or 6 singular values account for 70-80% of the sum of squares of our data. Now let's see how a rank-$k$ approximation does in cross-validation.\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def LrRankK_LogLoss(U, sig, Vt, Y_tr, cv):\n",
    "    '''\n",
    "    Runs cross-validation on each rank k approximation\n",
    "    '''\n",
    "    results = {}\n",
    "    \n",
    "    for k in range(len(sig)):\n",
    "        X_k = pd.DataFrame(np.matrix(U[:, :k+1]) * np.diag(sig[:k+1]))\n",
    "        mu, serr = runXVal_LogLoss(cv, X_k, Y_tr)\n",
    "        results[k+1] = [mu, serr]\n",
    "        \n",
    "        \n",
    "    return results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "r_svd = LrRankK_LogLoss(U, sig, Vt, Y_train, cv)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Now let's plot the results. We'll also plot it with the stepwise forward selection results above.\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEWCAYAAABbgYH9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsnWeYFNXSgN/aXWDJElVAFiQKEiRI\nEsWMCSMiClwwR/RT0ateJahXMGBAFMEAXBEMGDCBiiAoqyQJgiTJIAhIDrKhvh/Vw84OM7PL7szO\nhvM+Tz8zfU736erpnq6uc+pUiaricDgcDkekiYu1AA6Hw+EonDgF43A4HI6o4BSMw+FwOKKCUzAO\nh8PhiApOwTgcDocjKjgF43A4HI6o4BRMFBCRr0XkX7GWI9aISC0RURFJiLUs+RURmS4iN0ehXXcP\nHiMiUlNE9olIfKxlCUREOonIxii1vU9ETo5G2wVewYjIGSIyS0R2i8jfIvKTiLT26nqLyI95LZOq\nXqSqYyLZpvcgOuTdDL6lXSSPkdeIyFoRORhwTtVy2WbU/ohhjllDRCaKyHbvPvxNRHrn4fEHiMi7\n/mXRuAcjiYg8KiJrvGu+UUTe98pHiMjYINs3E5F/RKSid74pIrLXW1aIyKsicmKY4/UWkTS/+2yN\niLwjIvV926jqelUto6ppETg/38uV73hrReTfuW03AnId9ULjnfPqaByvQCsYESkHfAEMAyoC1YGB\nwD+xlCuK3O3dDL4l+Vh2FiMq1zwXVsplAee0OaKCHSM5PI//ARuAJKAS0BPYGkm5ChOeZdUTOE9V\nywCtgKle9RjgKhEpHbBbT+ALVf3bW39fVcti//srgROAeeGUDJDsHa88cB5w0Nvn1EicVwiO8455\nDfC4iJwfxWPlP1S1wC7YjbkrRN0pwCEgDdjn2w4oATwPrMceAiOAkl5dJ2Aj8CiwHVgL3ODV1QZ2\nAXHe+ijgL7/j/Q+4z/s+HbjZ+14X+AHY7bX5vt8+DYFvgb+B5cC1Yc71SJtB6toDc7xjzAHaB+z3\nNPAT9oe6BVjsV/8tMMdvfSZwhff938AfwF5gKXCl33a9vTZfBHYATwHx3m+7HVgN3AUokBBC7rXY\nQyZYXVtglvebLwQ6+dX1AX735FoN3OaVl/bOMd275vuAasBo4Cm//TsBGwPkeBhYhL2cJHj7TQS2\nAWuAvmGuzT6geZj6cOeS6boCN3rnthOYAiT51TX2u1+2YvdpZ+AwkOLJsTDIPRgH/AdYB/wFjAXK\ne3W1vGv0L+w/sR14LMR5tAG2APF+ZVcCi7zvpwNzgT2efENDtPMq8FKY32s50MtvPR7YDFzurQ8A\n3g3YJ977bZ8P0WZv4Mcg5V8AHwX8Fgl+v+FT3rXbB3yOvUCM885xDlArxPEyteWVzQb6+a2HvMeA\nkth9uxP77/Uj8z2rQF2/9dFkvscvBxZ4cv7h3SdPY8/DQ975vBrYFqZ8x3oyrfPumzj/3xD7j+/0\nZL4o1HVU1QKvYMphD7cxwEVAhaxuKuyBOAl78ynr3TTPeHWdgFRgKKaIzgL2Aw28+vVAS78/wWrg\nFL+604L8uccDj2F/8kTgDK+8NPbW2wd7oJ2G/bkbhTjXI20GlFf0LnZPr53u3nolv/3WYw+nBO+c\nDwGVgWLYg2CTV14Se0D79u2K/QnigG7eb3Gi32+bCtzjtVsSuB1YBpzkyTWNHCgYzBLdAVzsHft8\nb72KV38JUAcQ7xodAFr4XcONAe2NJmsFs8CTu6R3zHnAE0Bx4GTvWl8Y4jy+w5TtdUDNYzwX/3vl\ncmAV9nKUgP25Z3l1ZYE/gQew+6gs0MarG8DRD1z/dm/02j0ZKAN8DPzPq6vlXaNR3rk3w5TsKSHO\n9Q/gfL/1D4F/e9+TgZ7e9zJA2xBt9MCUZD/sJTE+oP4x4Du/9QuxB16xUOfrlQ8CfglxzN4EVzA3\nAlsDfgt/BbMKu9fKYw/6FZj1k4A9iN8JcbzAttpi9+mV3nrYewwYjL3sVcTuy9/IpoLBFP1u7F6L\nw+7BhqGeI2RWMGOBz7D7q5Z3vjf5/YYp2EtqPHAHpvgl5DM63AO8ICzYn3E0ZnmkYsrj+GA3FfZA\n2g/U8StrB6zxvnfy2ijtV/8B8Lj3/X/A/Zg5vhx4FnuoBlo3Ry6id8FGAjUC5O4GzAwoewPoH+I8\np3s36C5vme+V9wRmB2ybDPT2229QQP1M4Crvpv/GO8fOwNl4b6MhZFhAxltkb2B9QP33wO1+6xeQ\ntYLZ53dOn3rlD+M9AP22nQL8K0Q7nwL3+l3DnCiYG/3W2wQ5t0cI/TCpgD0QlmBviAuA1tk5l4B7\n5Wu8P7O3Hudd8yTsxeHXEMcfQHgFMxW406+uAfagSCDjQVjDr342cF2IYz0FvO19L4v9n5K89RlY\nF3XlbPxvb8AU835M4T7sV1fTk6+Gtz4OeDnc+XrltwMrQxyvN8EVTGcgxfvu+y38Fcxjftu+AHzt\nt34ZsCDE8Xxt7cJe2hR785fs3GOYsunsV3cr2VcwbwAvhpDryH0R2BamNA7j95IL3AZM9/sNV/nV\nlfL2PSHUdS7QYzAAqvq7qvZW1RrAqdgb90shNq+C/SjzRGSXiOwCJnvlPnaq6n6/9XVem2BdXZ2A\nM7E/03TsDfosTFmkBznmQ5himy0iS0TkRq88CWjjk8OT5QZMeYWir6oe5y0tvLJqnoz+rMPeWnxs\nCKj3P48fAs7jB99GItJLRBb4yXcqZvmEardaQFmgXMG4wu+crvDKkoCuAb/NGcCJnlwXicjPnlPH\nLsw6qBy8+WzjL3cSUC3g+I8CxwfbUVV3quq/VbWxt80C4FMRkazOJYAk4GW/7f7G7p3q2FvsHzk8\nt8B7ZB2mXPzPZ4vf9wOYBRKM97AxkhLYS8p8VfW1fRNQH1gmInNE5NJQAqnqOFU9DzgOUwxPisiF\nXt167P/VQ0TKAFdgL2pZUR37zY6FrPbxH0s7GGQ91O/ko7K3zQPYf66YV57VPZaT/5KPnN4rvl6N\nwHvF/1ly5D5R1QPe15C/QYFXMP6o6jJMk/sG7TRgk+3YTdHY76FWXm0QzkeFgAHGmpgZCPbw7Yjd\nKD9g/ZEdCHgwB8i0RVVvUdVq2NvAayJSF7t5fvCT4zi1Qe47jvG0N2M3qz81sW6vI2IE1AcqmB8I\nUDAikoR1m9yNdZkdh5npEqbdP7Gb21+OnLABe+v3/21Kq+pg78E2EXsbPN6T6ys/uQJlAntLLuW3\nHkyJ+++3AbNq/Y9fVlUvzkpwVd3uyVYN694IeS4hzvu2gG1Lquosry6UK2mwc/Yn8B6piVnqx+yI\noKpLsYfORcD1mMLx1a1U1e5AVWAI8FGQwfrA9lJU9UNs/Mt/sH0MZp1fjV2LeeHa8ZxXLsOs82Ph\nyhzsc0yoapqqDsW6pu/0irO6x7L6Lx0g9D29AevWCypOGFG3Y5Zj4L2yKfjmWVOgFYyINBSRB0Sk\nhrd+EtaV8LO3yVaghogUB/AsjFHAiyJS1dunuu/NyY+BIlJcRDoCl2L9zKjqSkxB9cCUg28w82pC\nKBgR6eqTDxsbUWwQ+gugvoj0FJFi3tJaRE45xp/hK6+d60UkQUS6AY289kMxC+smOR3rXluCZ1Fh\nb45gY0SK9X0jIn3I/AAIxgdAX89ttwLmJJAT3gUuE5ELRSReRBI99+MaWH91CU+uVBG5COuK87EV\nqCQi5f3KFgAXey6uJwD3ZXH82cBeEXlYREp6Mpwqnvt7ICIyxKtPEJGyWN/0KlXdkcW5BDICeERE\nGnvtlheRrl7dF8CJInKfiJQQkbIi0sbvnGuF8RAcD/yfiNT2LIL/Ys4mqVn8DqF4D7gXe0H50O93\n6CEiVbz/2S6v+CirXsxl+BLvHOK8a9gY+MVvs4nYw20gpmyC4v3mp3jneAI2fhoW7zrUFpFh2IvW\nwKz2iRCDgYdEJJGs77EPsHuhgnev3BPQ1gLgem+/ztjLoY+3gD4icq73+1YXkYZe3VZCvKiouWd/\nADztXZskbEjg3WDbZ4cCrWAwL6I2wC8ish9TLL9h5ijYmMASYIuIbPfKHsYG7n4WkT1YP3ADvza3\nYIpgM9b3e7tnGfn4Adihqhv81gWYH0LG1p58+7DxoXtVdbWq7sUejNd5x9qCvfWVOJYfwHuIXeqd\n8w6sS+5S70061D77PXmXqOphrzgZWKeqf3nbLMX6nJOxm7IJNpAdjlHY+MJCr/2Pj+Vc/OTbgA14\nP4opkg3YgHCc97v1xf4IO7G36El++y7DHjarva6HatjY2UJsrOUb4P0sjp+G/abNMU+Z7cCb2EBv\nMEoBn2AP1dWYsu6S1bkEOe4n2D0wwbs3f8MsBbzzPh97S98CrMTGzCDjIb9DRILdh297v8EM73wO\ncfQD61gYjz3Qvg+4zzoDS7x7/WVsHOdgkP33YL/Heuw3exa4Q1WPzFnz7tGJQA3sfxhIN+84u7Hr\nvwNzwAnn5t7O22cP1i1cDhsrW5z1KUeEL7F79pZs3GMDMUtxDXbP/i+grXuxe8HXtf6pr0JVZ2PO\nQy9iv88PZFglLwPXiMhOEXkliIz3YBb/aqyH5j3s/skRvgEnBzZJDxs8DPZ26XA4HI5joKBbMA6H\nw+HIpzgF43A4HI6o4LrIHA6HwxEVnAXjcDgcjqhQJMKoV65cWWvVqhVrMRwOh6NAMW/evO2qWiXr\nLYNTJBRMrVq1mDt3bqzFcDgcjgKFiBxLBIGjcF1kDofD4YgKTsE4HA6HIyo4BeNwOByOqBDVMRgv\nRs7LWBjoN4MF+BORa7Hw24olS7reKx+C5f0AeFJVfSlVx2E5JFKweD63qWpKNM+jqJGSksLGjRs5\ndOhQrEVxOBx5QGJiIjVq1KBYsWJZb3wMRE3BiEg8MByLn7QRmCMik7wYV75t6mE5EDqo6k6/AJSX\nAC2wOD0lgOki8rUXXHIcFmwSLE7OzcDr0TqPosjGjRspW7YstWrVQkSy3sHhcBRYVJUdO3awceNG\nateuHdG2o9lFdjoWUXa1F1BxAhb0z59bgOGquhPAF2gRiwY8Q1VTvaB3i7BAeqjqV+qBWTBRjRs2\nYEA0W8+fHDp0iEqVKjnl4nAUAUSESpUqRaXHIpoKpjqZE+ZsJHPiGrDkRPVF5CexBFKdvfKFQGcR\nKSUilbGosf65ERCRYli+iMlRkd5jYF4F8s5nOOXicBQdovV/j/U8mASgHpaToQYwQ0SaqOo3Xl6E\nWViI82QsFa0/r2FWTtBkQSJyK5ZmlJo1c5r3yuFwOBw5JZoWzCYyWx01ODoz2kZgkpfVbg2wAlM4\nqOrTqtpcVc/H8q2s8O0kIv2xNMf3hzq4qo5U1Vaq2qpKlWObiDpgAIjYYsezpSh2l8WKp59+msaN\nG9O0aVOaN2/OL7/8wsCBA3nkkUcybbdgwQJOOcVytNWqVYsmTZrQpEkTGjVqxH/+85+QZn98fDzN\nmzfn1FNP5bLLLmPXrl1Bt8sOnTp1ynIi7+jRo7n77ruz3KZKlSo0b96cRo0aMWrUqGOSY+7cufTt\n2/eY9snPBLsHAF566SUOHDiQxd45Z9KkSQweHCzh6LHhu8d8y9q1a3MvXC6ZPn06l14aMpN15FHV\nqCyYdbIaqI1lIVyIpSr236YzMMb7XhnrUquEeZ1V8sqbYomXErz1mzHLpmR2ZWnZsqXmFMjxrrmi\nf//YHFdVdenSpbE7uKrOmjVL27Ztq4cOHVJV1W3btummTZt0+fLlWrt27UzbPvzwwzpw4EBVVU1K\nStJt27apqurevXu1e/fu2qtXr6DHKF269JHvvXr10qeeeirH8p511lk6Z86csNu88847etddd2V7\nm61bt2rlypV1y5YtmbZJSUnJsZwFiVD3gGrm65yf8b/HjoVIXuPU1NRM69OmTdNLLrkk6LbB/vfA\nXM2FHoiaBaOWjvVuLMPh78AHqrpERAaJSBdvsylYFr6lwDSgn1qGxmLATK98JNBDM9K7jgCOB5JF\nZIGIPBGtc4glBW3sJzkZnnnGPnPLn3/+SeXKlSlRwpJ7Vq5cmWrVqlG/fn0qVKhw5E0W4IMPPqB7\n9+5HtVGmTBlGjBjBp59+yt9//x32eO3atWPTJjOu9+3bx7nnnkuLFi1o0qQJn332GQBr167llFNO\n4ZZbbqFx48ZccMEFHDyYOVljeno6vXv35j//+U/Y43355Ze0a9eO7dtDJh2latWq1KlTh3Xr1jFg\nwAB69uxJhw4d6NmzJ4cOHaJPnz40adKE0047jWnTpgExeDsNJII3Qah74JVXXmHz5s2cffbZnH22\nJfT85ptvaNeuHS1atKBr167s27cPMIv2oYceokmTJpx++umsWrWKtLQ0ateujaqya9cu4uPjmTHD\nsoSfeeaZrFy5MpO1+eGHH3LqqafSrFkzzjzzTADS0tLo168frVu3pmnTprzxxhvZPq9Q12706NF0\n6dKFc845h3PPPZe77rqLSZMsUeuVV17JjTfeCMDbb7/NY489BsAVV1xBy5Ytady4MSNHjjxyjDJl\nyvDAAw/QrFkzkpOTmTx5Mg0bNqRFixZ8/HGOkszmmKiOwajqV1jOeP+yJ/y+K9bNdX/ANocwT7Jg\nbebpuFH//nl5tPzHfffBggXht9m9GxYtgvR0iIuDpk2hfKjkwkDz5vDSS6HrL7jgAgYNGkT9+vU5\n77zz6NatG2edZSnHu3fvzoQJE2jTpg0///wzFStWpF69ekHbKVeuHLVr12blypW0adMm6DZpaWlM\nnTqVm266CbD5AJ988gnlypVj+/bttG3bli5d7H1o5cqVjB8/nlGjRnHttdcyceJEevQwj/nU1FRu\nuOEGTj311CMPgGB88sknDB06lK+++ooKFSqE3G716tWsXr2aunXrArB06VJ+/PFHSpYsyQsvvICI\nsHjxYpYtW8YFF1zAihUrQraVa2JwE4S6B/r27cvQoUOZNm0alStXZvv27Tz11FN89913lC5dmiFD\nhjB06FCeeMIeM+XLl2fx4sWMHTuW++67jy+++IIGDRqwdOlS1qxZQ4sWLZg5cyZt2rRhw4YN1KtX\nj59+ysgMPmjQIKZMmUL16tWPdKO+9dZblC9fnjlz5vDPP//QoUMHLrjggqNcfA8ePEjz5s0BqF27\nNp988gnDhw8Pee3mz5/PokWLqFixIhMmTGDmzJl06dKFTZs28eeffwIwc+ZMrrvuOsCUTcWKFTl4\n8CCtW7fm6quvplKlSuzfv582bdrwwgsvcOjQIerVq8f3339P3bp16datW/jrGGHcTP4syMtxl4I6\n9rN7tz1XwD53785de2XKlGHevHmMHDmSKlWq0K1bN0aPHg1At27d+Oijj0hPT2fChAlBrRd/NES+\nI9+f/4QTTmDr1q2cf/75R7Z/9NFHadq0Keeddx6bNm1i69atgD0kfA+Mli1bZupTv+2227JULt9/\n/z1Dhgzhyy+/DKlc3n//fZo3b0737t154403qFixIgBdunShZMmSAPz4449HFFvDhg1JSkqKroLJ\nDhG+CcLdA/78/PPPLF26lA4dOtC8eXPGjBnDunUZ8Rl990f37t1J9iyrjh07MmPGDGbMmMEjjzzC\njz/+yJw5c2jduvVR7Xfo0IHevXszatQo0tLMz+ibb75h7NixNG/enDZt2rBjxw5Wrlx51L4lS5Zk\nwYIFLFiwgE8++QQIf+3OP//8I9e7Y8eOzJw5k6VLl9KoUSOOP/54/vzzT5KTk2nfvj0Ar7zyCs2a\nNaNt27Zs2LDhiAzx8fFcffXVACxbtozatWtTr149ROTIsfOKWHuR5Wu+/RbmzIGzz4Z27aJ/vAED\nMpSJCOSHXHDhLA0fyclw7rlw+DAULw7jxuX+94qPj6dTp0506tSJJk2aMGbMGHr37s1JJ51E7dq1\n+eGHH5g4ceKRh0Yw9u7dy9q1a6lfv/5Rdb4//4EDB7jwwgsZPnw4ffv2Zdy4cWzbto158+ZRrFgx\natWqdcRRwNdd45PPv4usffv2TJs2jQceeOCIFTTQ6+d88803AahTpw6rV69mxYoVtGrVKqjM3bp1\n49VXXz2qvHTp0tn41aJEjG6CUPeAP6rK+eefz/jx44O24e9+6/t+5pln8vrrr7N582YGDRrEc889\nx/Tp0+nYseNR+48YMYJffvmFL7/8kpYtWzJv3jxUlWHDhnHhhRfm6vwC8b/GPotp8uTJnHnmmfz9\n99988MEHlClThrJlyzJ9+nS+++47kpOTKVWqFJ06dTpynyYmJhIfHx9R2XKKs2BCkJwMF10Ejz0G\nZ5wBN90EX38NWXTnF0natYOpU+HJJ+0zt8pl+fLlmd4IFyxYQFJS0pH17t2783//93+cfPLJ1KgR\nfJ7tvn37uPPOO7niiivCdkWVKlWKV155hRdeeIHU1FR2795N1apVKVasGNOmTcv0NhyOm266iYsv\nvphrr72W1NRUrrzyyiNvrz5lkpSUxMSJE+nVqxdLlizJVrvB6NixI+PGjQNgxYoVrF+/ngYNGuS4\nvYgQ4Zsg3D1QtmxZ9u7dC0Dbtm356aefWLVqFQD79+/PZM29//77Rz7beTKdfvrpzJo1i7i4OBIT\nE2nevDlvvPHGkTEWf/744w/atGnDoEGDqFKlChs2bODCCy/k9ddfJyXFIlStWLGC/fv3Z+u8juXa\ntW3blpdeeokzzzyTjh078vzzzx9Rgrt376ZChQqUKlWKZcuW8fPPPwdto2HDhqxdu5Y//vgDIKQi\njhbOggnB9OkZFkR6OrzzDrz9tq3Xrw9t2kDbtrY0aQIRDuFT4MZ+2rWLnJW3b98+7rnnHnbt2kVC\nQgJ169bNNIjZtWtX+vbty7Bhw47a9+yzz0ZVSU9P58orr+Txxx/P8ninnXYaTZs2Zfz48dxwww1c\ndtllNGnShFatWtGwYcNsy33//feze/duevbsybhx44iLO/r9rWHDhowbN46uXbvy+eefU6dOnWy3\n7+POO+/kjjvuoEmTJiQkJDB69Ogj1lVMJ8hG8CYIdw/ceuutdO7cmWrVqjFt2jRGjx5N9+7d+eef\nfwB46qmnjlitO3fupGnTppQoUeLIw7VEiRKcdNJJtG3bFrCH/vjx42nSpMlRcvTr14+VK1eiqpx7\n7rk0a9aMpk2bsnbtWlq0aIGqUqVKFT799NNsnVe4axdIx44d+eabb6hbty5JSUn8/fffRxRM586d\nGTFiBKeccgoNGjQ4ci6BJCYmMnLkSC655BJKlSpFx44djyjnvEBC9VEXJlq1aqXHmnAs0OL//HOI\nj4eff85YvK55EhOhVavMSifEi3WB4Pfffz8yt8RRcJg4cSKTJk1izJgxsRYlX+BLNFi5cuVYi1Ig\nCPa/F5F5qhq8PzcbOAsmBD6Lf/p06NQp48WsUyf7VIX1603R/PKLfb76KrzwgtVXq2aKxqd0WraE\nWHajOwo3kyZN4rHHHuNtn5ntcOQDnAUTQQ4fhoULMxTOzz+D1/VJfLx1pfkrnfr1zaMzv+EsGIej\n6OEsmHxO8eLQurUtvqgg27bB7NkZSue992DECKsrXz5D2bRpY0ulStY9F2g55TWq6gJeOhxFhGgZ\nGk7BZMEnn1j3Vk7jZVapApdcYguYw8Dy5Zm71p56KmMKQfXqsGWLrScmRsYr61hJTExkx44dLmS/\nw1EEUC8fTGJiYsTbdgomDLt3w7XXQmqqTTzu0gUuuwxatMh511ZcHJxyii19+ljZvn0wb54pm7Fj\nwYtawj//mCWT1wqmRo0abNy4kW3btuXtgR0OR0zwZbSMNG4MJguWLzcPskmT4KefzLJ4+WXo29cU\ngKpZGpHC573mm8P37bdw3nmRa9/hcDiyS27HYPLhEHP+okEDePBBmDHD3JLHjIHLvbycn34KlSvD\nVVfB6NE23pJbfN5rd9xh635hkRwOh6NA4SyYXPDrrzBqlFk3mzZZeJf27eHLL8PH+csuXbta9IA/\n/oDjj899ew6Hw3EsOAsmhpx2Grz2GmzYYGMo/ftD1apQrpzVP/JIhvWTmhq+rWA8/TQcOmTRNxwO\nh6Og4RRMBBCxgf/+/eHjjzOiIa9bB8OGwVlnmQXSqxd88032261fH269Fd54A4IEa3U4HI58jVMw\nUeS992D7dvjoI7j0UvjqK5g82epSU20+zPr14dt44gkoUQKyyGHlcDgc+Q43BpOHpKbCgQPWhZac\nbOM1kOEC3aWLWUKBU0/694dBg2zCZpCUFQ6HwxEV3BhMASIhIWN8pl07WLYMnn0WypSxyZatWtl4\nDYB/9O8HHrAJmw89lD9yxDgcDkd2cAomhjRoAP36wcyZNnt/zBizatLSYOBAiyIAppQef9wmXU6Z\nElORHQ6HI9u4LrJ8SGpqRlfZr7/a7P/Dh232f5kyMH++Bc90OByOaOK6yAohCQnWHbZokU3mBAuk\n+fTTVvbee7GVz+FwOLKDs2DyKamp0LgxlCxpFktcnIWpOf10ixiwfHlkQ9Q4HA5HIM6CKaQkJNi4\ny8KF8NlnVhYXB0OGmGvza6/FVj6Hw+HICqdg8jHXXWd5ZRo0yCg791y48ELrLtu1K3ayORwOR1Y4\nBZOPSUiwSACNGmUuHzwYdu40a8bhcDjyK07BFACWL4d//ztjDkzz5nDDDfDSSxm5YxwOhyO/4RRM\nAeCXX8xamTQpo+zJJ23Qv3//2MnlcDgc4XAKpgBw/fVQty4MGJBhxdSqBXfeCe+8A0uXxlI6h8Ph\nCI5TMAUAn0fZggWZrZjHHrOJl488EjvZHA6HIxROwRQQglkxlSvDww+b0vnxx5iK53A4HEcRVQUj\nIp1FZLmIrBKRf4fY5loRWSoiS0TkPb/yISLym7d08yuvLSK/eG2+LyLFo3kO+YWEBIuo3KmTJSHz\ncd99cOKJLhCmw+HIf0RNwYhIPDAcuAhoBHQXkUYB29QDHgE6qGpj4D6v/BKgBdAcaAM8KCJeHGKG\nAC+qal1gJ3BTtM4hv9G9O7z4os3u91GqlAXGTE7OmJDpcDgc+YFoWjCnA6tUdbWqHgYmAJcHbHML\nMFxVdwKo6l9eeSNghqqmqup+YBHQWUQEOAf4yNtuDHBFFM8h36EK339vEZh99OkDDRvaWExOUjM7\nHA5HNIimgqkObPBb3+iV+VOKcNvxAAAgAElEQVQfqC8iP4nIzyLS2StfiCmUUiJSGTgbOAmoBOxS\n1dQwbQIgIreKyFwRmbtt27YInVLsSUuzNMr33pvRJZaQAM88Y/ll3nkntvI5HA6Hj1gP8icA9YBO\nQHdglIgcp6rfAF8Bs4DxQDKQdiwNq+pIVW2lqq2qVKkSWaljSEKCpU/+9Vf44ouM8ssvt1wy/ftb\n1kyHw+GINdFUMJswq8NHDa/Mn43AJFVNUdU1wApM4aCqT6tqc1U9HxCvbgdwnIgkhGmz0NOjB9Sp\nk9mjTMQmY/75p83wdzgcjlgTTQUzB6jneX0VB64DJgVs8ylmveB1hdUHVotIvIhU8sqbAk2Bb9Ry\nC0wDrvH2/xdQ5Ia2fVbM/PmZrZgzzoAuXUzRbN8eO/kcDocDoqhgvHGSu4EpwO/AB6q6REQGiUgX\nb7MpwA4RWYopjn6qugMoBsz0ykcCPfzGXR4G7heRVdiYzFvROof8TI8e1iW2f3/m8meegX37LNqy\nw+FwxBKXcKwAo2pdY4HcfDOMHWtBMmvXznu5HA5H4cAlHCvCiJhb8iefZJ5kOXAgxMfDE0/ETjaH\nw+FwCqaA88EHcNVV8OWXGWXVq9sM/3HjLH6Zw+FwxAKnYAo4XbtaN5i/RxlYjLIKFSyPjMPhcMQC\np2AKOMWKmUfZvHnw1VcZ5ccdZ9GWp0yBqVNjJ5/D4Si6uEH+QkBKCjRoAJUqwezZGQP///xj5ZUr\nW3mce51wOBzHgBvkd1CsmFkrBw/CX39llJcoYZkv582zsRqHw+HIS5wFU0hISzPLJdBKSUuDFi1s\nbszvv0PxIpHcwOFwRII8tWBEJM4vbL4jHxEfb8plzx5YsSJz+ZAhsHo1vPFG7ORzOBxFjywVjIi8\nJyLlRKQ08BuwVET6RV80R0445xyb5e9vmF54IZx9tiUs27MndrI5HI6iRXYsmEaqugfLu/I1UBvo\nGVWpHDnm9tthzhz4+uuMMl8gzO3b4YUXYiebw+EoWmRHwRQTkWKYgpmkqilA4R+4KaD06gW1ah09\nL6Z1a7j2WlMwW7aEb2PAgCgK6HA4igzZUTBvAGuB0sAMEUkCXEdLPqV4cfMomzMHJk/OXPf00+a6\nPGhQ+DYGDoyefA6Ho+iQpYJR1VdUtbqqXqzGOizDpCOf4rNivvsuc3ndunDbbTByZGZHAIfD4YgG\n2Rnkv9cb5BcReUtE5gPn5IFsjhxSvLjNfQk23vL441CypFk5/gwYYGM1vkmavu+uu8zhcOSU7HSR\n3egN8l8AVMAG+AdHVSpHrqlY0T43b848FnP88fDgg/DRR/DLLxnlvjEb37a+707BOByOnJIdBePL\nOHIx8D9VXeJX5sjHzJxpXWVTpmQuv/9+qFoVHnoos/JxOByOSJIdBTNPRL7BFMwUESkLpEdXLEck\naNMGTjzxaI+ysmWhf3+YMSOzO7OP/v3zTESHw1GIyTJUjIjEAc2B1aq6S0QqAdVVdVFeCBgJikKo\nmFCMHGkD+5Mn24RLHykp0KgRJCZazpj4+NjJ6HA48idRDxWjqulADeA/IvI80L4gKZeiTu/eULPm\n0VZMsWLw3//Cb7/Bu+/GSjqHw1GYyY4X2WDgXmCpt/QVkf9GWzBHZPCfF7N0aea6a66xCZiPPw6H\nDsVGPofDUXjJzhjMxcD5qvq2qr4NdAYuja5YjkjSu7fNe2ncOHO5CDz7LGzYAK++GhPRHA5HISa7\n0ZSP8/tePhqCOKJH8eJw8sn2/cCBzHWdOsFFF1l32c6deS6aw+EoxGRHwTwD/Coio0VkDDAPeDq6\nYjmiwW23wXnnHe2aPHgw7Nplnw6HwxEpsjPIPx5oC3wMTATaATOjLJcjCrRoAcnJ8O23mcubNoWe\nPeHll627zOFwOCJBjjJaish6Va0ZBXmiQlF2U/bn8GGLR1ajBvz0U0ZYGIB166B+fbjhBnj77djJ\n6HA48g95mtHS/7g5PaAjdhQvDo8+GtyKSUqCe+6B0aPhvvtsG4fD4cgNOVUwLsBIAaVPHzjpJBg6\n9Og63/jMyy9bBkynZBwOR25ICFUhIsMIrkiEzF5ljgJEiRIwcSLUq3d03a+/QlwcpKdb3pgBA+Cr\nr9wsf4fDkTNCKhgg3KCFG9AowLRubZ/p6ZlD9HfqZAro8GFb/+Ybs2TGjrWgmQ6Hw3EshOwiU9Ux\n4ZbsNC4inUVkuYisEpF/h9jmWhFZKiJLROQ9v/JnvbLfReQVEXsMikh3EVksIotEZLKIVD7Wk3bA\n6tXmVTZ1akZZu3a2/uSTFghz7FiLU9asmYWTcZGXHQ7HMaGqYRfgc2BSwPI/LHxMYpj94oE/gJOB\n4sBCoFHANvWAX4EK3npV77M98JPXRjyQDHTCLK6/gMreds8CA7I6h5YtW6ojM4cOqdaoodqhg2p6\neujt1qxRPeMMyw7TrZvq339HT6b+/aPXdn4+tsORXwHmahbP13BLdgb5VwP7gFHesgfYC9T31kNx\nOrBKVVer6mFgAnB5wDa3AMNVdaen7P7y6T0g0VNMJYBiwFZs/EeA0p5FUw7YnI1zcARQooR5lP30\nE3z/fejtatWC6dPh6adt7KZp0/Db54aBA6PTbn4/tsNRWMmOgmmvqter6ufe0gNorap3AS3C7Fcd\n8J+2t9Er86c+UF9EfhKRn0WkM4CqJgPTgD+9ZYqq/q6qKcAdwGJMsTQC3gp2cBG5VUTmisjcbdu2\nZeM0ix433mhzYgIjLQcSH5/h3lyqlHmb9etnjgAOh8MRiuwomDIicmRSpfe9jLd6OJfHT8C6yToB\n3YFRInKciNQFTsHSBFQHzhGRjiJSDFMwpwHVgEXAI8EaVtWRqtpKVVtVqVIll2IWTkqUgEcegR9/\nNCslK1q1gvnz4fbb4fnn4fTTLdx/bhgwILOjge97XqRqjuWxHY6iQHYUzAPAjyIyTUSmY2FiHhSR\n0kC4wf5NwEl+6zW8Mn82ApNUNUVV1wArMIVzJfCzqu5T1X3A11iImuYAqvqH1z/4ATZe48ghN91k\nkyvPOCN725cuDa+9Bp9/Dn/+aUrn5ZfNIy0n+KwnnwXl+55XCiZWx3Y4igLZiUX2FfbQvw8b2G+g\nql+q6n5VfSnMrnOAeiJSW0SKA9dhDgL+fIpZL3jeYPWxMZ/1wFkikuBZLWcBv2MKqpGI+EyS871y\nRw4pUQL+9S9LQHYsXHopLF4M559vM/8vugg2u9Ewh8PhR3YSjhUDbgMe95abvbKwqGoqcDcwBVMC\nH6jqEhEZJCJdvM2mADtEZCk25tJPVXcAH2EeaIsx77OF3vjPZmAgMENEFmEWjUt+FgHGjoVrrz02\nV+Tjj4dJk2DECJg5E5o0gY8/zrkM/fvnfN/cEstjOxyFlSyDXYrIm5gXl687rCeQpqo3R1m2iJGr\nYJcDBhSJPpPhw+Huu20ezDnnHPv+y5dDjx4wd66Fo3n5ZShbNvJyFjaKyO3lKKDkNthldhTMQlVt\nllVZfibHCmbPHihfvkjMMDx0COrUseWHHzJHWs4uKSnm7vvMM+be/L//QXs3QhYWkdjdXk65ObIi\nL6Ipp4lIHb8Dngyk5fSABQZVuOwy+75nT2xlyQMSE82jbOZMmDYtZ20UKwZPPWUKKj0dOna0rqeU\nlMjK6ogMbu6PI9pkR8H0A6aJyHQR+QH4HvMsK7wMGGBRH2fMsPXy5YuE/+rNN9u8mFtuyZ1SOOMM\nWLjQkpgNGmTrK1dGTs6CjnOPdhQZsjPdH5tN39RbSgBtchM+IK+XXIWK8XmvPvdcztsoQCxZovrp\np/Y9PV01NTV37X3wgWqFCqqlSqmOHBk+LE1RBPL2eP37Z9zS/ktehspxIYEKDuQyVIzLaJkVInDN\nNeYe9d13Fl64iPDmmzZHZvx4yyGTUzZtMlfoqVPh8sth1Chwc1+NWI7BxOrYRfGcCyouo2W06d/f\ncgjXrw/dusHGjbGWKM8oW9a6upo3t4mVOaV6dQv9P3QofP21uTN//XXk5CzIOPdoR2HGZbTMigED\n7En78cdw8CB07ZqRMKWQ062bhYapWRO6dIH778/5qcfFwf/9n7kxV60KF19sbtEHDkRW5oJGLMdd\n8lK5uZBARZOQXWQi8jmhM1qeo6qloylYJMlVF5k/H31kCuauu+DVV3PfXgHh0CELbvnqq9ZLeO65\nuW/v0UfhxRehYUMYN85y0ziKBq6LrOAQtXkwInJWuB1V9YecHjSviZiCAXjwQXjhBZv63rNnZNos\nICxebN1bAKtWQd26uWvvu+9sbGbbNvNgq17dJnm2a5d7WY+F5GQL9tmpU94fuyjiFEzBIbcKJmTK\n5IKkQPKUwYOtn+e22yzVY9OmsZYoz/Apl/nzoU0bc2ceOtTm0OSE884zpXXNNfD661YWF2cP+QoV\nbD3YhM/AsqzWw22zY4dFk05Ph+LFTellN/CnI2e4kEBFhxx5kRU0ImrBAGzdan06JUuasjnuuMi1\nXQA4fBgee8xC9jdrBu+/Dw0a5Ly9//4XHn88IyJztWpwwgnB3zQDy7Jaz2qbbdvscvpITLTxps6d\nbTnxxOyfh8NR2Ih6qJjCQMQVDMCsWXDWWRZG+NNP7dW7iPHll9bFdegQvPEG3HBDztpJTrZxncOH\nzYqYOjXvuqr8jx0fb1bV/PmwZYvVN2uWoWzatzf5HI6iQqzclIsGyckWWCs5+ei69u1tlPrzz22b\nIsgll8CCBXDaabB6dc7badfOlMqTT+atcgk89vTppjQ3b7bzGjzYuupeeMGmP1WqBFdcYdGj167N\nOxkdjoJKTrzIAFDVLqHq8hs5smBmzbJR39RU6zcJ9uRTtRDC48fD5MlwwQURk7kgkZpqYxrx8RaH\nrFIlOPXUWEsVOfbuhe+/t0v89dewbp2VN2iQYd2cdZb1mDochYloWjDPAy8Aa4CDwChv2Yflainc\n/PCDPTlVLfl8sJzCIjByJDRuDN27F9nX2oQEUy7p6XDPPZZK+a23Co+3TtmyFoHg9ddhzRpYtgxe\neglq17auwYsugooVTdG89JLVF5ZzdzhyQ3bC9c8N1GDByvIzObJgfJ3zBw/a+uefWxrHYKxcabmD\n69Uzl6SculUVArZsMaNu6lS4/nrrTirMeWEOHrSYqJMn27JsmZUnJZnCuegic70uzL+Bo/CSF2Mw\npb0Q/b4D1gYKzCTLHOPrnL/vPhvZfeWV0Inn69WzeTHz5tkrfBHmhBNgyhQL2z9hgjnb/fVXrKWK\nHiVLwoUX2nDc77+bhTNihIXXGTfOxmwqVrQxnCFDLPSOs24cRYXsWDCdgZHAamwWfxJwm6pOib54\nkSHXXmQjR9q8l2eftSntoXjsMfO5ffNNuOmmnB+vkDBzprkwDxuWswRmBZ3Dh20oz2fdLFxo5See\naErp5JNh3z5TPh07QokS1t3ocOQX8sRNWURKAA291WWq+k9ODxgLcq1gVC1EzGef2ROjdevg26Wl\nWb/IzJnw00/QsmXOj1nIWLXKcsO88kqRmzZ0hM2bLejn5Mnw1VfmPBBIXJwpmsRE+/RfAsuys02o\n/f74w5YuXcyXxeEIRl6kTC4F3A8kqeotIlIPaKCqX+T0oHlNRObB7NxpkyKKF7eJEuXKBd9u+3br\nF4qLsy6zSpVyd9xCwnvv2ZyZGjWs66xNm1hLFFuefhqeeMJ6XePizAGxfXvzJ/FfDh3KuizYNsfS\nDVehgvXyJiVZquukpMzf3fhR0SUvFMz7wDygl6qe6imcWaraPKcHzWsiNtHyxx/NH/X66y3hfChm\nz7Y+j7PPtokV8fG5P3Yh4Jdf4LrrLOPB4MEWXbkIzk8Foju5VNUykoZSTCNH2pKebl2XLVuaklm3\nzpZ/AvonKlY8Wun4K6IKFYpmF2hRIC8UzFxVbSUiv6rqaV7ZQlVtltOD5jURnck/aJAFNMoq2KVv\n3Obxx20fBwC7dtnw1McfW3dZUfaJiFWQzXDKLT3dnDLWrs1QOL7vvs/9+zO3V7ZseAVUtWqGAopl\nYFEX1PTYyQsFMws4F/hJVVuISB1gvKqentOD5jURVTBpaeZ3On++LfXqBd9O1Z6k77wT3sW5CKIK\n774LV18NpUrZG3OJErGWqmiR04etKvz9d3gFtGtX5n1KlrScQscdZ6H70tPNqO/Z08pFzJL1/wxW\nFq4uq7I//rCAG2lpeR+OqCCTFwrmAuAxoBHwDdAB6KOq03J60Lwm4rHINmyw8ZiTT7ZB/1ABqg4e\nhA4dLI7K3Lm5j29fCNm3D9q2ta6zRx5xvYmFgd27M5SPv+JJTrb02bEmPt5CAz3ySKwlyf/klRdZ\nJaAt5qb8s6puz+kBY0FUgl1+8glcdZXlh3nuudDbrVljndwnnWT/sFKlIitHAWfvXutJHD/enPPq\n1oVixczbu3p1C6jw2WdW5lsSEqxrrVw509tz51qZf/0VV5jeX7nS3gd85b5tmja1N9sdO+w94MQT\nnXKLNqG65lTNqlHN/D3wM1xdVtvMn28vMapmUTkLJnvkVsGgqmEXYGp2yvLz0rJlS40Kd9xh9/Tk\nyeG3++orVRHVnj1V09OjI0sBJj1d9c03VZs0Ua1bVzUpSXXVKqsbNky1TBnVxETV+HjfI0R10yar\n798/o8x/2b3b6h94IHh9WprV33qrrZ9wguqdd6p+/71qSkpe/wJFh1mzVP/7X/vMax5+2K7166/n\n/bELKsBczcWzN1ywy0SgFDAN6IRZLwDlgMmq2jDojvmQHFswK1bY5Mk334Ty5Y+uP3jQXru3bYNF\ni+D440O3NXCgJQF/7TW4445jl8UB2NtoaqpZISI24Lxnj5WlpNiSmgqnnGL972vWwPr1metTUsz4\nBJuutGiRBbP88ku7pKedZm+8YOrIeUgVDvbvNzf5Cy80V3lH1kQzZfK9wH1ANWATGQpmDzBKVQtM\nUvocK5hffrHJCb17W/TGYPz2mymZs86y2XOh/G7T0+Gyy+Dbby14Vdu2xy6PI6rs32+TIPfvh169\nbEC4WTML3nnNNZYrxuWDKdg88IB5L65bZ4ntHOHJiy6ye3JjIuWHJVddZI8+anb155+H3ub1122b\n558P39aOHaq1a6tWr666dWvOZXLkCX//rdqjh2q5cnZ5y5e3Xs7582MtmSOnrFxpvdVPPBFrSQoG\nRKuLLECLnYp5kR0JE6yqY3Os1fKYXA3y//NPRjfYb78Fn5mvaj63X3xhXmWtwij8X381q6hdO4sb\n4oJP5Xv++Qe++w4mTrTkpRMm2Mz7lSth8WKLDuR8NwoOl1xiQTbWr3cWaVbkhQXTHxuH2Qq8A2wB\nPsqO9gI6A8uBVcC/Q2xzLbAUWAK851f+rFf2O/AKGd15xbHgmyuAZcDVWcmR60H+X39VTUhQvf/+\n0Nvs2KFao4aNUu/ZE7690aPtlfihh3InlyPPOXxYNTXVvj/xhF3GUqVUu3ZVnTBBde/e2MrnyJqv\nv7br9t57sZYk/0MuLZjsKInFWFj/hd768cC32dgvHktMdrKnFBYCjQK2qQf8ClTw1qt6n+2Bn7w2\n4oFkoJNXNxB4yvseB1TOSpaIeJFNmaK6f3/4bX74QTUuTrVXr6zbu+02+/k/+ij3sjliQkqKeZ3d\ncYfq8cfb5axcOcMLzaeIHPmLtDR7D2zfPtaS5H9yq2CyEwnqoKqmA6kiUg74CzgpG/udDqxS1dWq\nehiYAFwesM0twHBV3Qmgqr7MIYp1xxUHSgDFMAsK4EbgGW/7dM2rOTkXXGD9IPv32+SJYJx5Jvzn\nPxZG5t13w7f38svW9danT0aWKkeBIiHBws299ppNIJwxw6ZE+Xo927Y1v44xYyxWqiN/EBcHd91l\nvdk+b0FHdMiOgpkrIsdh6ZLnAfMxiyIrqgMb/NY3emX+1Afqi8hPIvKzl3sGVU3GuuX+9JYpqvq7\nJwfAkyIyX0Q+FJGgvsEicquIzBWRudu2bcuGuNkgNdWeGjfeGDpc7eOP2+z9O+6w+BShKFECPvrI\nPq+6yqa0Owos8fEW37R3b1tPSbH3jUWLrKxqVRur+fbbWErp8NG7t70vvlpgfGELJlkqGFW9U1V3\nqeoI4HzgX6raJ0LHT8C6yToB3YFRInKciNQFTgFqYErpHBHp6G1fA4vm3AJTdM+HkHukqrZS1VZV\nqlSJkLQJZnFMmhQ6mnJCgqUyTEiA7t1t2nIoata0EePly+Hmm12qw0JEsWLwwgsWJmX2bLj/fnMK\n+PNPq9+82TJfrlxp1s/WrRbjKy3N6n2z0B3R4bjjzBX9vfdCd0g4ck9IBSMiLQIXoCKQ4H3Pik1k\n7kqr4ZX5sxGYpKopqroGG7ivB1yJhaTZp6r7gK+BdsAO4ADwsbf/h0B2ZIkc995rr6p9+1oMkmAk\nJcGoUTBnjlk04Tj3XEsO8v771m3mKFSIWE/okCGWdO2GG6z866/NyK1f3yb/nXCCOSguX271r7xi\nXTnFillok3LlLGy+L5bXq69anNVTTrGwNy1aWI4dXxKzN9+Eiy6yhGJXXQXdukGPHhkK7P33Lfjj\nhx/CggXBk58Vdu66yzwEQ01xc+SecD6yL3ifiUArbJBegKbAXOyBH445QD0RqY0pluuA6wO2+RSz\nXN4RkcpYl9lqzDHgFhF5xjvmWcBLqqoi8jlm8XyPRXlemvVpRpD4eIuQ3KyZRUueMiX4VO9rroFb\nb7U0y+eea2M4oXj4YZvU+eCDFresY8foye+IGSIZ8c5uvNEmcM6blxF9ICXFFA2YsujfP3N0gpQU\nKF3a6qtVs/3961JTM9o/eNDezAMjGPjmAScnH/0+c/LJpgRFLKrB/v0WG65u3dD59Qoyp55q0aRf\ne80mYLpYdFEgKy8AzFpo4rd+Ktl3U74Ys0r+AB7zygYBXbzvAgzFlMRi4DrN8EB7A3NRXgoM9Wsz\nCZgBLAKmAjWzkiMqschee021QwebjReK/ftVTznFXIyymli5a5dqvXoWFGvz5sjK6nAEYe9e88D/\n8EOLD/b44xl1HTtqpthtVauqXn99Rv20aapz59ptW5D56CM7v08/jbUk+ROiPdFSRJaoauOsyvIz\nUYmmrJqR2CIcixbZq6Yvu2W4FI6LF5sTwWmn2fZPPhlZmR2ObLJ/v/morFpl40SrVkHlytatBlC7\nto0vgZXXq2cRrB96yMoWL7YA4scdF7T5fENqqp1Lw4bOASMYeZEPZjywH/D53d4AlFHV7jk9aF4T\nFQXjY+tWGD7cAlmGUh7Dh8Pdd8PQoZYnOBzjx1tKZnCjvI58y5IlFgt21aoMJdS+PTz1lI3zlCpl\n/i2VKpnyqVvXeo0vv9zGPWbNOjopWO3alqLh4EFYuPDo+qQkU2YHDmR05fkv1atbTNoDB2ysKjD5\nWNWqwSMu/Pe/FtN26VIb03JkkBcz+ROB/wM+8Zb/AxJzYzbl9RK1cP2qqmPHmo09dGjobdLTVbt0\nUS1WTHXevKzb7NvX2rz2WtV16yInq8ORBxw+rPrJJ6rPPWfpEM45R/Wkk1SfftrqN23K3P3mW3yh\n/JYtC17/xhtWP2dO8Ppx46z++++D10+aFFzerVtVixdXvfvu6P4uBRHyIhZZQSeqFoyq9Q1MmWJx\nxkK9Au3YYY4BpUvbyG6ZMkdvM2CAhfUPpFMncztKTDy6zuEoIKiX+uCff8zJwP/xn55uHnVJSdY9\nN2PG0fVNm0KtWjZp9fvvj1Yhbdva/lu2WOw4/31VLRr2SSGmiPfqZTkEN20qnA4NOSWa4fo/UNVr\nRWQxNrM+E6raNKcHzWuiqmDA7uhTT81IoRwqgOX06XDOOTbL6+23w7cpYp3cDz5oEzJr17Yutssv\ndwlKHI4csHcvjB5trttJSZnrZs82z71hw6w322HkVsGEm2h5r/d5KXBZkMXh44QT4PXXbd7Liy+G\n3q5TJ+vsfecdG2vJiqQkm6gwdap1Hl95pU0H//33iInucBQVdu2C++6DN944uu70022+0quvuqHP\nSBJSwajqn97numBL3olYQOja1SYW+GKFhKJ/fwvVf/vtsHp1+O18nHOOdb+9/LLNl2na1Bz3d++O\niOgOR1HgpJMsNtybb1o3XSD33GMTXadOzXvZCivhusj2EqRrDJu7oqpaYHoqo95FFkhqqnX8hko2\nsXYtNG9uvpEzZ9p07eyybRs8+qhNP65aFQYPtg7kcO7PDocDsBRMF15o0ZyuD5j2feiQRW9q397y\n/jii2EWmqmVVtVyQpWxBUi55zsGDNhM/2GC9j1q1LJTML79ktlSyQ5Uqtu/s2TYu06ePWUSzZ+dK\nbIejKHDeeVCnjvVoB5KYCLfcAp9/njHHx5E7sv3aKyJVRaSmb4mmUAWakiXNk2zwYFMgoeja1QJc\nDh6cM5u8VSv46SeLBb9+vY1Q3nSTzctxOBxBiYuzGHBxceatFsjtt5sPzYgReS9bYSQ7Ey27YHHJ\nqmG5YJKA37Woz+QPx+7d0KSJKZtffw2dT3f/flMUu3fbzLKcRn3es8dmuL30kh1zwABzhTmWrjeH\no4jgc5cOxTXXwLRpsHGj/Z2KMtH0IvPxJNAWWKGqtbEAkz/n9IBFgvLlzQ15xQrzGgtF6dIWrn/H\nDuvqyqn7SrlyFlRz8WLrQL7/fptz8913OWvP4SjE+JTL1q3BrZi777bUCRMm5K1chZHsKJgUVd0B\nxIlInKpOw6IrO8Jx3nkWD3zyZItdEYpmzeD55y1O2bBhuTtmgwbw1VeWr+aff+D88y1W+5o1uWvX\n4Shk/PGHeZWNHXt03VlnQePG9nd0Lsu5IzsKZpeIlMEiGI8TkZex2GSOrHj2WZu1H6qLzMfdd8Ol\nl0K/fpacIzeImC/mkiUWZGnKFGjUyJwJwik6h6MIcfLJ1ov92mtHKxER+0v++iv87PpqckV2FMzl\nwEEsBtlkLPS+m2iZHUqVsmX/fsvwFAoRm3xZqRJcd11wu/1YSUyERx4xx/4rr4RBg8wt+sMP3WuZ\no8gjYoP9v/1mvjKB9IZWogUAACAASURBVOhhPd0upXLuCJfRcriIdFDV/aqapqqpqjpGVV/xuswc\n2eXll01xTJkSepvKleHdd23c5t57Q293rNSoYXlhZ8ywlIjXXmsTNxcvjtwxHI4CSPfupkSCuSyX\nKWPDoh9+aJGgHDkjnAWzAnheRNaKyLMiclpeCVXouP9+c12+6SaLVxGKc84xq+Ott+zuT06OnAwd\nO1p33WuvWY6a006ztM87d0buGA5HAaJ0afjXv+Djj80RM5A777QsoCNH5r1shYVwEy1fVtV2WLri\nHcDbIrJMRPqLSP08k7AwkJhoo4lbtmRtnXTubE76EyZY7LJZsyInR3y89QusWAG33WZ5aurVs3+Q\nL1k7mJuzw1EEeOghywMTLIJyvXr2dxwxwhSN49jJcgzGiz02RFVPA7oDV2CpjB3HQqtW5rI8dmz4\nOBQ//pjx/fBhs9NXrIisLJUqmXKZP9/cZW67zaL9+ZRZuCgEDkchonp1C4gRirvvhj//tFD+jmMn\nSwUjIgkicpmIjAO+BpYDV0VdssLIY4/BrbdaaP9QdOoEJUqYtVGsmM32OvVUePhhizceSZo1sxQC\n48fbpIAOHaBnT6vzt2gcjkLMli3mxPnVV0fXde5sHmdusD9nhBvkP19E3gY2ArcAXwJ1VPU6Vf0s\nrwQsVBQvbrHC69bNyIYUSLt2FjrmySfhhx8s4nKPHuby3KCBOQJE0gtMxBwQevWy9Xe9zNgJCVbX\npg3873+WIsApHUchpFIlG54cPvzouvh4m842c6YF23AcG+GiKX8PvAdMVNUCPRKc56FismLPHhtd\n7Nr16JCuofjlF4snPmeOzdYfNgxatIi8bBs32gy0e++FuXNtMoBv/kzp0nbMVq2gZUv7rFfPRXJ2\nFHieeMKiLf3xx9FdZjt3WlfaDTdYnNmiRNQyWhYm8p2CSUuDM86AZctsQmS1atnbLz3dUvL9+9+w\nfbuFfn36aXNxjiQiGVZSWprJOXeuLfPmmdI5dMjqy5bNUDo+xVOnjlM6jgLFxo0W5LxfP3jmmaPr\nb73VjPuNG83bv6jgFEw2yHcKBmzgvnlzOPts+OKLY0uDvGuXDcQPG2YP+CeftDCwoVI1HysDBoT3\nJEtNNdebefMyFM/ChRlZnMqXN0Xjs3JatbLXQpfq2ZGPufJK87HZuNGGQf1ZuND+rs8/b7n+igpO\nwWSDfKlgwBRE376WYu+mm459/6VLbf+pUy3uxbBhFkgpFqSkmDXms3LmzrX5NocPW32FChkKx/eZ\nlJRZ6WSl2ByOKDJzps1Hvvdem2gZyJlnwqZN9m4YH5/38sWCqCkYEampqutD1HVU1Zk5PWhek28V\nTHq6BcVcv966oXJigaiaD+X998O6ddCtGzz3nI2jxJrDhy0Wh3/32qJFZgGBja76K52rr7bfxFk6\njnzIhx9aIIzPPzevs6JANBXMamAE8IKqpnllx2O5YRrm5qB5Tb5VMGCvRMWL5zwXjI8DB8zTbMgQ\nG/949FGz5RMTIyNnpDh0yMLU+KycefNs3eehVraszc1p3Njcs33fTzzRKR5H1Dl82N7Xmjc3p01/\nUlJsnKZJEwuSXhTIrYJBVYMuQAXgDWAxcA5wL7AOuAuIC7Vfflxatmyp+Z7UVNW5c3Pfzpo1qldd\nZU7QJ5+s+tlnqunpuW83WvTv73PYzryULJl5vUIF1TPOUL39dtVhw1SnTVP9669YS+8oZGzfrpqY\naLdZMAYNsttx+fK8lStWAHM1F8/e7GS0vBd4EdgMtFXVjTnWZjEiX1swPh5+2MZQFi4019/c8t13\nNj7z++82W+yll45+Jctv+HuvAfz1l43r/PZb5k//eG5Vqwa3eCpUyHv5HYWCPn3go4+scyEwhMyW\nLVCzpsUpe+ml2MiXl0Szi+w4YAjQBngIuBjLZnmvqn6f0wPGggKhYDZvtgdkw4Y22hiJUcSUFJs9\n5ssFc9998PjjwQMv5QcCFUwwVC12R6DSWbIE9u3L2K5atQyF4/ts1Mi64ILhHAwcHrNn2/zi4cNN\nkQTSo4eNw2zaFNwZoDARzS6y1cCDQIJfWXNgFjA+O+YR0BkLLbMK+HeIba4FlgJLgPf8yp/1yn4H\nXsFThn71k4DfsiNHgegiU1UdN87s78GDI9vuli2qffpY2yecoDp2rGpaWmSPEQn698/5vunpqmvX\nqn75peqQIaq9eqm2aHF0V1tSkurFF6s+9JDqmDGq8+apHjhgdbEgN+fsiArp6aotW6o2bhy8dzk5\n2W6X117Le9nyGqLVRSYiNTREd5iI3KKqYee0ikg8FvL/fCzczBygu6ou9dumHvABcI6q7hSRqqr6\nl4i0B54DzvQ2/RF4RFWne/tdBVwDNFXVMIG9jAJhwYA9Art2tdejuXNtNDGSzJ5t0QBmz7aQNMOG\nmfdWYSYtDdauPdriWbYsw4XaZznVrGnWXdmy4T/D1ZUocWzOCNmx2hx5zltv2ZyX6dPh+OMz16lC\n69Zw8KDdToXZ9yS3FkxIv9hQysWry07AhNOBVaq6GkBEJmDZMZf6bXMLMFy9UDSq+pfvEEAiUBwQ\noBiw1WunDHA/cCumnAoPIpb96K+/ohMf/PTTLcfMmDEWDaB1a7j5ZosGkFsvtvxKfLxFFqhTBy6/\nPKM8NdW6DIcPz3jAr/e88mvXNsW0fr2F9dmzx7rfsqMIEhKyp6R8n2DjbnXrWigeR77gX/+CG28M\nrjx8KZX79DEFdPbZeS5egSFqEy1F5Bqgs6re7K33BNqo6t1+23yKWTkdgHhggKpO9uqeB27GFMyr\nqvqYV/4iMAP4FfgilAUjIrdiSoiaNWu2XLduXVTOM6ocPAhffmnzQyL9mrR7t0UDeOUVe9gNGmS5\nYiIVDaCgkZUlkZ5uqaz37jWFk5vPUFGxy5Qxi7JevcxLnTpQsmR0ztsRlv377dIHDt0dPGhTzc46\nCyZOjI1seUHULJg8IgGoB3QCagAzRKQJUBk4xSsD+FZEOgJ7sYjO/ycitcI1rKojgZFgXWTRED7q\njB5to4wdO1p3VrNmkWu7fHkYOtQsmL59bRk50kLO7NljaQPatYvc8Qo6cXH2lClbNvux40LhU1Z7\n9lhK6/ffh5UrM5bPPoNt2zLvc9JJRyueunVN+QTGNXFEhL//tp+3Xz+bVuZPyZIWCvDZZ83QrVkz\nNjJmxfROA+g0fUDsBMjNAE64BWgHTPFbfwQbR/HfZgTQx299KtAa6Ac87lf+BObJdgfmLr0WG9c5\nDEzPSpYCM8gfSFqa6qhRqpUrq8bFqd5xh+qOHZE/Tnq66sSJ5gDgGwxPSFAdOlR19+7IHy8/EqvB\n9lDOBbv+v70zj5KqvvL451dLL7QIQoMIyLTiEgEBEXGhQWXRRoOKGie4BI/GbeYkgnI8ISajMpKY\nqIEkasQ4LlEjceGcoIkYgiBgWkV2EYUOYaQFkaVRtt6q7vxxq6aqNyjofvWqq+/nnHfe8ntd777u\n6vd9v/u79/52iyxdKvLHP4o88IDI9deLnH22SOfOdYMWnNPAhVGj9Pvxq1+JvPGGyKefilRVHfza\nbTHA4DDvecQIkV69NE2tPps26b/lj3986M9ZcP7hXbfFaGbwCl7nwRwpzrkQ6v4aCXyBDvJfKyJr\nk84pQQf+JzjnClG310BgFDo+U4K6yOYCM0TkjaSfLeIgLrJkWs0gf1NUVGio8eOPwyWXaBCAF0yd\nqqG6yd+JQEDTmocP16W4OHvHa/zgSMKjd+2CsrK6vZ74kpwjFAxqvbf6PZ+TT9aU9HDYnwADP0PC\nDzOo4vXX4eqrYc4cGDu2YXu8QObmzYcomnGw60ajOuZaXa3rI9luqi1eeb1z55Tvua7ZGVzs0jl3\nCTADHV95RkSmOeemoqo4xznn0NIzJUAEmCYis2IRaE+gUWQCzBWRu+p9dhFtRWDirFmjD/y+fTXj\nq6xMH/gtRWkpjBypX86cHC07s2OHVgB8//1Eif4+fdRtFxednj0P/rlGehCBnTsbF54NG+qO/QSD\nGsjQpYu62LxY8vIaP37KKWpPNKpLJJLYPtR+c8+97TYdd6ytTSyRSN39pGOR6lpmvVBL1061jB7R\n8Lwd22pZ9mGE/n1qOa7wIJ/52Wfq5mxMCKLRtHw9Fp5/32G7yzJaYDKFrBGYZCZN0lTi665TR3Bz\nxwXilJZqaEz9MZiqKg2dXrRIE0GXLEk8sE44ISE2w4bp2EA2x262RkQ0OnHKFHj22Ybt/fpppYeq\nqtSXeJh3thAMapBL8hIM8s2BEBV7QvToFSKUW/ccCYVY9XEQgiEGDk78DKEQO94vo3DnZw0u80X3\ns+hxzVB9iQuHE0vyfkttFxQ0q4CsCUwKZKXA7NsHDz2klZPDYc3QnzhRv1zpoLZWKyMvWpQQnR07\ntK1bt4TgDB+uPS6bgCzzaG4OTjSqIpOKGD33HMya1fAzrrpKSxQHg/odCQSa3j5YW6rn9eihvbwk\nISAU0rYmHsLbtqkLbHATj9knntBplT/4QDMBGsWvfKdmXtcEJgWyUmDibNyovZk5c7Rk/6OP+mOH\niCYvxgXn3Xe1lgZoXbDi4oTgnHGGiqLhL630oZdp196zR3XriivgD39I33VToblRZK09TNloLiee\nqGGtc+cmMv/jMyL17p0+O5yD007T5bbb9J9p06ZE72bRokRwQkEBnHdewqU2ZAisXNm4a87wjvvu\n89uC9HOE97xnj05EVlKiHa5k2reHG2+EmTM1+79r14Y/v/D8+7jgiK7cPHwNUcZ6MNnJpZdqNeXJ\nkzWAP1MyxLduTYjN4sXqYgN1UUSjKkrhsArRRRf5a6vhHa2wsGg0qnVou3SB995r2P7pp/pu9eCD\ncO+96bfPK8xFlgJtTmC2bNHy/y++qBFejzyir12ZNvC+a5f+tz78sApOMgMGqFutuFh7OT16+GOj\nYcSYPl290CtXNp7zfNFFOov5pk3ZUxCjuQJjI6/ZSPfu8MILGunVpQt897vw9NN+W9WQTp00ueAX\nv9DU6GBQw1hvvlntfu45GD9eRfKEE+CGG7TawCefpC200zDiTJigkde/+13j7T/4gQ47/vnP6bUr\nk7EeTLYTiejI4zXXqKts9WqNx8+0CbkaC4+urdVCkPGw6MWLNdQWVJziPZziYq3hla4IOqPNctNN\n8Mor6iSoP61SJKIR+kVFsGCBL+a1OOYiS4E2LTDJRKOaJLlzJ/zsZ/rf0hITm6ULEU0uXbIksaxf\nr215eTpLVFxwzjsvcydWM1oty5drMfKf/KTxghYPPwz33KPvcS0924YfmMCkgAlMEitXamHLxYv1\nrf+3v23dUVvbtuk4TryHs2KFvkoGAtC/f2IMp7i45ZJRDaMJdu5Uj+6ECfDkk35b03xMYFLABKYe\nIpr0Nnmy9vUXLtS649nA3r2a8RZ3q5WW6nTRoOM4cbEpLtawIOearl5gGI0gol+XwsLGeynf/z68\n/LKOx3TsmHbzWhQTmBQwgWmCvXvhmWd09qRAQN/++/XLriTImpq64zhLliTGcTp3Vpfh++9rrycc\n1mCIkSN1jMfK4BuNsH+/BjWWlKiQ1GfFChg0SKPOJk5Mv30tiQlMCpjApMDXX+voZLduWq+qpKTx\njLHWjogWWoyLzZw56tdojIICFZrOnVNfH3NMdgm00SiTJmlx882bG06pDNpB3rZNa1y25ipJJjAp\nYAKTIm++qf85ZWW6f+aZOmqZzXPClpbCiBHa0wmFdPS2sFBFZ9eupteRSNOfefTRqQnSli0qdpdf\nDkOHpu+ejWbz2WfqYZ02reFkZKAe6PHj4a9/hTFj0m9fS2ECkwImMIdBNKp9/LlzdZk+Xav8zZun\ntTBKSuDiizXUOVs43DEYEZ2N8mAC1Ni6oqLpelT9+mnkW//+6tg//fTMCyU36jBqlL4fbNzYMBiz\nulqn4hk0SGc9b62YwKSACUwL8NJLOnlRebnu9+2rYjN1KrRr569trYVoVCcE27VLp6ueOTNRSr2o\nSNsqKhLn9+ypQhMXnf79taS+5ftkBK+/rlWUFy3SKW7q88ADuqxfr/kxrRETmBQwgWkhRDSLPt67\nKSvT1zfn1CEtoqLTWv+b0kn9yd3mz4dzzlG32erVOrncmjW6vW6duvBA3Xjf+lZd0Tn9dBWjTCsF\nlOXU1ur7QVN6v3Ur9OqlWQF+FTlvLiYwKWAC4xG1tYmiSyNGJNKXe/dWobnqquwev2kuqbrmamrU\n6Z8sOmvWwOefJ87p2DHhWouLTr9+lmyaBiIROHAAjjqqYdv48Tq0OXmy1ipLRxS8iNq06slSds1e\nSNdrLmDA7Ud2YROYFDCBSRNlZfD229q7eecduP56dQOJaELniBHqWrM37ZZh9274+OO6orNmjY4P\nxSkqauhmO/lkWLq07eX+eJDvVFWlke5XXaUTy9Zn5kx47vZSLmQhi4PnM2ziYHp1rUT2HyC6vxI5\nUAkHDui6shJXeUDXVZUEqg4QqK4kUF1JqOYAgZpKQjWVhGoPEKqtJFxbSThygJxIJeFoJbmRA+RI\nJblSSUcq6MaXOKCSPMpmvsPptx7+PZvApIAJjA9UVekkGoWF6lbr21ePxxMISkpg9Gjo0MFfO7MN\nEe3ZJIvO6tXaA4pHvoXD2vsU0Rjac8+F447TvJ+cnLrrxo4drO1Q53/0kU5Gd8EF6hKMRNTPFI16\nu71mDdx9dyJa8L/+S+dSqq5OLDU1jW8fou3jlTXs313N4P7VBGrrtu3fuY/8A4lxtea8WkVxVAfy\nqQ7mURPMoyaUT20oT5dwPpFwHpFwHtGcfHI3b6D33lUEEGoI8t5F/80Fb0857GuawKSACUwGUF6e\n6N3Mm6d5N2+8Ad/+tiYTbNumITetOWkgk6mq0klLVq+G3/++7vQIxx2nEWtVVXWnQI5vt9XK1fG5\n7XNyEkvyfmy7Yl8OS1fn0GdAmJ4n1G3b88FaCj5ZSgAhgmPnwFEUXFlCsCBPl6PyCbTL01p6+fm6\nbmo7HE6597/mqVJ63zaSMNXUkMM/Z863HoxXmMBkGLW1Ws5l4EBNZpw6VWca7NIFLrkELrtMHdaN\nObWN5tNYgMHBXEaRSEPRaUyIGjuWvD1vni4i+qAcMUJLFAWD+mIRCHi3vW5dogcTr9hw1llNi0go\nlPLDXETjLjp10l9t/d915MLE7zq44BC/6xZkzVOl7Hx9IZ2vuuCIxAVMYFLCBCbD2b4d/vY3TRh4\n6y0dW+jYUUu6hMNam8NCoVsWP+qvHa6weXF9j+55xgzNUW50MrJWXOvOBCYFTGBaETU1Wh25rEyr\nBoK+adbU6ORkY8dq4qe50lonrfhhezAqKvTWLr64dc2AcShMYFLABKYVI6JJiXPmaO2waFTrpd17\nrxbpNAzDM2zKZCO7cU595+++qy6zF16A4cMT0WdbtmigwMyZWh/dMHyipkbrxD7/vN+WZA7WgzFa\nN6Wlmm+zcaPuDxqkQQK33954mVvD8JAhQ3QWjLVrsyPdy3owRtvm3HN1vGbtWvj5zzWcc+pUjVQD\ndav95S+aam0YHnPHHRqwtmiR35ZkBtaDMbKPiopEJeKrr9aqhPn5mtg5dqy61Lp189dGIyvZv1/L\nwo0eDX/6k9/WNB/rwRhGfZLL3L/0kiZ33nSTxpDecosKTJxNm5ouoW8Yh0m7dnDjjTB7Nnz5pd/W\n+E/Iyw93zpUAvwaCwNMi8lAj51wD3A8IsEpEro0d/yVwKSqC84A7gXzgVaA3EAHeEJEfeXkPRisn\nN1djRy++WOuhrVmjeTagr5unnaYJngMG6JhNt25axqa4WN1sGzfq8aOPzg6nuuE5t9+ulZTNK+uh\nwDjngsDjwGigHFjqnJsjIp8knXMyMAUYKiIVzrmusePnAUOB/rFTlwDnAx8Cj4jIAudcDjDfOTdG\nRN7y6j6MLMI5LfaYzGOP6bSDGzfCsmUaqdahgwrM55/r/CugYztxAZoyRWeh3L4dXn01cTy+tgoE\nbZpTToGXX/bbiszAyx7MEKBMRDYCOOdmAZcDnySdcwvwuIhUAIjIV7HjAuQBOWh9uDCwTUT2Awti\n51Y755YDPT28ByObadcObr5ZlzjRaCJAoHNnePFF9XVs25ZYxycAWbdOZ5yqzyuvwHe+o4L14IMq\nOskCNGyYfna8ZIqRlaxdq6HLAwf6bYl/eCkwPYDNSfvlwNn1zjkFwDn3HupGu19E5opIqXNuAbAV\nFZjHRGRd8g865zoCY1EXnGG0DIFAQkA6dIDrrmv63KFDVXTqC9AZZ2j7119rhNuSJbBjR+Ln/vEP\njX574QW46y6tTDBkiFYsGDLEwquzgEhEvbL9+ukQYFvF0zGYFK9/MnAB2hNZ5Jw7HSgETiPRO5nn\nnBsmIosBnHMh4GXgN/EeUn2cc7cCtwL06tXLy3sw2irBoIpBU4IwYoSO+YC+ym7frgIUn1/31FNh\n3Didm2XatETV4s2bNRRp2TJNqjjzTHO7tTKCQY0nuf9++Oc/dQ6+toiXUWRfAMcn7feMHUumHJgj\nIjUi8i9gPSo444D3RWSviOwF3gKSCxc9BWwQkRlNXVxEnhKRwSIyuEuXLi1wO4bRDMJh6N5dezcF\nBXrs7LO1dP7KlTpJ2OLFGojQo4e2T5+uNbs6dNBX4Ztu0vONVsEtt6jQPPmk35b4h2d5MLFexnpg\nJCosS4FrRWRt0jklwHgRmeCcKwRWAAOBUej4TAnqIpsLzBCRN5xzD6K9m++ISEoTVVgejNEq2b5d\nezcffphYd+uW6BVNmqTjOHH32kkn2ZhOhnH11Vrbs7xc40RaGxld7NI5dwkwAx1feUZEpjnnpgIf\nicgc55wDHkWFJAJME5FZsQi0J4Dh6ID/XBG5yznXEx3X+RSoil3mMRF5+mB2mMAYWYGIJpF26qT7\nY8dqyft4POwxx8Ctt8JDsWyAXbsS5xq+8M47+mf6+9/VI7p8ecNJN88+G7p21VJ6771Xty0a1aj5\nY4+F9ethwYKk9ojgqqsYf9k+CvP3sWpDO95aWohUVVO0YR7hqr2ceu2ZnD7upCO2P6MFJlMwgTGy\nltpanZL6ww91GTBAI9v279fcnR49EsEDQ4boeE779n5b3WYQgW921dIhvJ+F8yNceKUmAQ/hAzqy\nmwL2cd/dexnQex//2FLE0AfHADCDOzmGCgrYx4VD9tEpvJcV3S9l0KtTCBBhF50oYB8hIv9/rRUj\nJzNo/sO05xu+QYvBLvr3xxk+6z+O2H4TmBQwgTHaHHv26KyNceGJFwOdPh0mTtSR5yee0Ii53NzE\nbI7jxumIdHm5+naS23JztZhox46arLplS922nBwVtXRMiCKiSyRS95U/HgyxZ4/27JLbAI6PDQt/\n9plG/VVVQWWlrsNhuOIKbX/1Ve0yxNsqKzUh96c/1fbJk9VVmfzzfftqWDtolODq1doWu3bNyBI+\nuP8tAgE4c9zx5H5VXueWasZeyYaHXicQgBPHnIKrqUYKjiJ0dAGB9gVUXzyWnd+bRCAARz9wt/rc\nCgrI7XwUgfYF1PYdQO3AwQSIElj+EcGjC3A9e+jf6whprsD4HUVmGIYXtG+vYzRxduzQcZw+fXS/\nvFynOKiu1gi3OH36qMB89BHccEPDz333XZ0u4c03G29ftkxF6Kmn4Ic/rDsVcU6OilZRkbb/8pcN\n/UWrVkFhoUbVPfJIoi2+3r1b68pNnAi/+U3daweDiRymO++EZ5+t296xo7oYAX7yE3jttbrtxx+f\nEJhnntH4Yuf0QZ6Xp4EWcYHZuVPD0HNzNQgjNzchXqBCXVysP5ebC/n5hE86ieLiWPvsWfrZBQUq\nigUFhNu3p08s/oN/rW/wq80BjovvPPFog/YQ8Qd6AM4b0qDdD0xgDKMtUFgIY8Yk9s8/X0OgQXsC\nNTX6Fh4fiR49Wt/gq6t1qarSdbwSwrBhWs0xua26OvGQ7d9fBS65raoqEUHXvTucc47mHQUCKg7J\nOUgDB8L3vle3LRhM9I7GjNFk1frtca6/PjHzaf3PBhWYO+7Qh39cQJKn5Z49G0IhXRoLnKgvXvW5\n556Dtw8devD2LMFcZIZhGEajWDVlwzAMIyMxgTEMwzA8wQTGMAzD8AQTGMMwDMMTTGAMwzAMTzCB\nMQzDMDzBBMYwDMPwBBMYwzAMwxPaRKKlc2478L9+23GYFAI7DnlWdmH33Dawe249/JuIHPGEWm1C\nYFojzrmPmpNB2xqxe24b2D23HcxFZhiGYXiCCYxhGIbhCSYwmctTfhvgA3bPbQO75zaCjcEYhmEY\nnmA9GMMwDMMTTGAMwzAMTzCByTCcc8c75xY45z5xzq11zt3pt03pwjkXdM6tcM696bct6cA519E5\n95pz7lPn3Drn3Ll+2+Q1zrlJse/1x865l51zeX7b1NI4555xzn3lnPs46Vgn59w859yG2PoYP21M\nFyYwmUctcLeI9AHOAf7TOdfHZ5vSxZ3AOr+NSCO/BuaKyLeAAWT5vTvnegA/BAaLSD8gCHzXX6s8\n4TmgpN6xHwHzReRkYH5sP+sxgckwRGSriCyPbe9BHzo9/LXKe5xzPYFLgaf9tiUdOOc6AMOB/wEQ\nkWoR2e2vVWkhBOQ750JAO2CLz/a0OCKyCNhV7/DlwPOx7eeBK9JqlE+YwGQwzrki4AzgA38tSQsz\ngHuAqN+GpIkTgO3AszG34NPOuQK/jfISEfkCeAT4HNgKfC0if/PXqrRxrIhsjW1/CRzrpzHpwgQm\nQ3HOHQW8DkwUkW/8tsdLnHPfBr4SkWV+25JGQsAg4Hcicgawjyx3m8TGHS5HxbU7UOCcu95fq9KP\naG5Im8gPMYHJQJxzYVRcXhKR2X7bkwaGApc55zYBs4ARzrkX/TXJc8qBchGJ905fQwUnmxkF/EtE\ntotIDTAbOM9nm9LFNufccQCx9Vc+25MWTGAyDOecQ/3y60TkV37bkw5EZIqI9BSRInTQ9x0Ryeo3\nWxH5EtjsnDs1KNfmEQAAAWZJREFUdmgk8ImPJqWDz4FznHPtYt/zkWR5YEMSc4AJse0JwJ99tCVt\nmMBkHkOBG9C3+JWx5RK/jTI84QfAS8651cBA4Gc+2+Mpsd7aa8ByYA36/Mm6EirOuZeBUuBU51y5\nc+5m4CFgtHNuA9qTe8hPG9OFlYoxDMMwPMF6MIZhGIYnmMAYhmEYnmACYxiGYXiCCYxhGIbhCSYw\nhmEYhieYwBiGDzjnipKr7RpGNmICYxiGYXiCCYxh+Ixz7sRYwcuz/LbFMFqSkN8GGEZbJlYqZhZw\no4is8tsew2hJTGAMwz+6oDWprhSRbK9DZrRBzEVmGP7xNVoAsthvQwzDC6wHYxj+UQ2MA952zu0V\nkT/6bZBhtCQmMIbhIyKyLzbh2ryYyMzx2ybDaCmsmrJhGIbhCTYGYxiGYXiCCYxhGIbhCSYwhmEY\nhieYwBiGYRieYAJjGIZheIIJjGEYhuEJJjCGYRiGJ/wfa7CCR1FTBBEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.clf()\n",
    "\n",
    "k_svd = []\n",
    "mu_svd = []\n",
    "serr_svd = []\n",
    "\n",
    "for i in range(len(r_svd.keys())):\n",
    "    k_svd.append(i+1)\n",
    "    mu_svd.append(r_svd[i+1][0])\n",
    "    serr_svd.append(r_svd[i+1][1])\n",
    "    \n",
    "plt.plot(k_svd, mu_svd, 'b.-', label = 'SVD Rank-k Proj')\n",
    "plt.plot(k_svd, np.array(mu_svd)+np.array(serr_svd), 'b+')\n",
    "plt.plot(k_svd, np.array(mu_svd)-np.array(serr_svd), 'b--')\n",
    "\n",
    "\n",
    "plt.plot(ks, mus, 'r.-', label = 'Stepwise Forward')\n",
    "plt.plot(ks, np.array(mus) + np.array(serrs), 'r+-')\n",
    "plt.plot(ks, np.array(mus) - np.array(serrs), 'r--')\n",
    "\n",
    "plt.legend(loc=1, ncol=2)\n",
    "\n",
    "plt.title('Stepwise Forward Feature Selection vs SVD Dim Reduction')\n",
    "plt.xlabel('k')\n",
    "plt.ylabel('X Validated LogLoss')\n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>This is a very interesting plot. We can see that SVD approach does almost universally worse than the stepwise approach. Its not until we get to the last two features do we see them equal each other. <br><br>\n",
    "My interpretation of this is that the SVD approach does not reduce the feature space with the outcome or learning task in mind. It is purely untangling the covariance structure of the matrix $X$. It is quite possible (based on the evidence above), that the most predictive features corresponded to the lowest singular values. That is why we don't see a large reduction in the cross-validated error until the last few columns are added into the matrix.<br><br>\n",
    "\n",
    "This is not to say or prove that dimensionality reduction with SVD is a generally inferior approach. Because the matrix $X$ is dense and not very high dimensional, the SVD approach just might not be appropriate here. It is also slower, as the cost of generating the SVD isn't always cheap.\n",
    "\n",
    "\n",
    "</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
